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PREFACE 


Evaluation has become a commonplace word and practice in development 
programmes throughout the world. Not only are programmes, plans and 
activities being designed and implemented to promote development in 
education, health, agriculture, etc. but they are also being evaluated regularly 
and frequently. There has also evolved, over the last couple of decades, 
specialized expertise, institutions and a body of knowledge in support of 
evaluation. However, much of this evaluation practice and theory con- 
tinues to be utilized in a manner that is very similar to top-down models 
of development, which are currently being replaced by bottom-up 
strategies. 


In the course of our work with grass-roots organizations and non- 
governmental initiatives in India, and other countries of Asia, we have 
evolved, through our own practice, some ideas, principles and theory 
about Participatory Evaluation. We evolved these principles not only 
based on the critique of traditional evaluation approaches but also based 
on what seemed relevant and appropriate in a given concrete situation. 


In our interactions with other practitioners, we have found multiple 
and varied meanings being given to the concept of Participatory Evaluation. 
For some, Participatory Evaluation implies the process of action-reflec- 
tion-action. In the process of conscientizing and organizing the poor, 
several grass-roots activists and organizations have evolved a methodol- 
ogy of action-reflection-action. It is this process which is equated with the 
meaning of Participatory Evaluation by some. On the other hand, we have 
been calling our involvement in facilitating evaluation and analysis of a 
programme, an organization or a perspective as Participatory Evaluation. 


This collection has arisen out of that practice and reflection, and limits 
itself to our definition of Participatory Evaluation. Over the years, many 
development activists, grass-roots organizers, leaders of non-governmental 
organizations, nationally and internationally, and managers of development 
programmes in government have been asking us about our interpretations, 
meanings and principles of Participatory Evaluation. This collection is 
intended to assist in answering those questions. Itis essentially a compila- 
tion of our own involvement in Participatory Evaluation with several 
types of development programmes and organizations in India and 
elsewhere. 


The bulk of the material in this volume is case-studies— eight of them. 
There is an opening section which essentially outlines our understanding 
of participatory evaluation. What is Participatory Evaluation? Why is it 
needed? Why is Participatory Evaluation needed in development? And, 
briefly, how is such an evaluation carried out? After the case-studies, the 
last section attempts to respond to a series of questions that have been 
raised in the context of our own practice over these years. It is hoped that 
these responses would provide beginnings of an answer in the future. 
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Therefore, this volume can be of some assistance to those who are 
planning to get involved in similar exercises in their own development 
efforts; it may provide examples for others for discussion and critique; it 
may raise some issues which others have also been grappling with in the 
course of their practice of Participatory Evaluation. 


We shall appreciate your responses and contributions to further this 
process in refining the theory and practice of Participatory Evaluation in 
development. 


The process of evaluation is 
integral to human thinking, 
reflection and daily existence. 


INTRODUCTION 


The growth of human civilization has implied human reflection through- 
outits history. As human beings, individually and collectively, confronted 
nature for their basic survival in the early days, they also began to reflect 
on their experience in order to improve ways and means of finding food 
for survival. This process of reflection is what has now come to be known 
as ‘evaluation’. Thus, evaluation is an integral process of human develop- 
ment and existence. All of us engage in evaluation of our day’s work, of 
the different stages in our life, of our accomplishments, of our failures, as 
well as, an evaluation of others and their accomplishments. Thus the 
process of evaluation is integral to human thinking, reflection and daily 
existence. 


With the rise of specialization in different areas of work and disciplines, 
evaluation has also become a specialized activity. This has become par- 
ticularly so in the context of development theory and practice. Over the 
last two decades, development programmes involved in agriculture, rural 
development, health, education, forestry, drinking water, etc. have been 
evaluated with great vigour and rigour. Evaluation has also been incorpo- 
rated as an integral part of the planning and implementation of each 
development programme. Thus the practice and usage of evaluation has 
become widespread in all development initiatives throughout the world. 


By and large, most development programmes now provide for time and 
resources for evaluation and invite those specialized in conducting such 
evaluations to do so. As a consequence of this practice of evaluation, 
certain distortions in the process have emerged over the years. First of all, 
evaluation became a specialized, separate activity away from the ongoing 
planning and implementation of a development initiative. It became such 
a‘ specialization that specialized people and institutions began to be in- 
volved in it; and programme planners and implementers and those be- 
nefitting from it felt alienated from this process of evaluation. 


Secondly, misplaced emphasis on separation of programme implemen- 
tation and evaluation got supported under the belief of objectivity. It was 
felt that those implementing a development programme may not have the 
necessary motivation to assess their efforts dispassionately and objec- 
tively. Thus external people and institutions, mostly outsiders to the 
given development programme, began to play the central role in evalua- 
tion. 


A third distortion emerged as a consequence of resource-providers, be 
they national or international donors, demanding that evaluation of the 
programmes they support be carried out periodically. Thus evaluation 
became the link to the possibilities of future resources and, therefore, 
something to be cautious of and to protect oneself from. As a consequence 
of evaluation exercises becoming the links to the question of continuation 
of grants and resources, it became an exercise in “hiding the mistakes” 
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Participatory Evaluation is the 
methodology of making 
evaluation an integral process 
of any planning, and 
implementing a development 
initiative which puts people 
involved in it in the centre and 
not remain on the periphery. 


and “putting the best foot forward”. Thus, when externally appointed 
evaluators visited programme sites, everybody praised the programme, 
kept critical information away from surfacing and remained generally ina 
state of terror during the period evaluation was being conducted. 


Thus evaluation became a tool to control programmes, resources, prog- 
ramme planners and implementers in the field of development over these 
years. This alienation and’ distancing of the evaluation process, from 
being an integral part of human thinking and activity, provided the basis 
for posing questions and evolving an alternative theory and practice of 
evaluation. This is what has come to be known as Participatory Evalua- 
tion. 


Participatory Evaluation is the methodology of making evaluation an 
irttegral process of any planning, and implementing a development initia- 
tive which puts people involved in it in the centre and not remain on the 
periphery. This implies a collective process of reflection, critical assess- 
ment and review about the accomplishment (or lack of accomplishment) 
of programme goals. Therefore, having a standard, a goal or an objective 
as a yardstick becomes important in determining the value of a given 
activity. 


Historically, evaluation has implied assessing the accomplishment of 
the goals of a given programme, by measuring the impact of the activities 
and plans when the goals were set. 


Clearly this kind of an evaluation can help us understand the extent to 
which the planned activities led to the accomplishment of the goals, the 
reasons for not meeting these goals, and the consequences of such ac- 
tivities other than accomplishment of the stated goals. The value of evalu- 
ation can be extended if it can be seen as an exercise in promoting the 
future development of the programme and the people involved in it, and 
not merely a historical analysis of the past. Thus, this link between the 
past and the future is one of the starting premises of Participatory Evalua- 
tion. By definition, Participatory Evaluation is intended to be develop- 
mental in nature, and not regulatory or controlling. It is intended to 
promote the growth and development of programmes, plans, perspec- 
tives and organizations in the future. Therefore, it is seen as an interven- 
tion within an overall framework of the past and the future. 


CHARACTERISTICS OF PARTICIPATORY EVALUATION 


The central characteristic of Participatory Evaluation is that people involved ina 
given development programme or organization, both as implementers and as be- _ 
neficiaries, start participating in, and take charge of the evaluation efforts. _ 


The control over the process of evaluation remains.in the hands of those who are _ 
developing and implementing and benefitting from the programmes. Thus, the 
evaluation serves the interest of furthering the benefits and i \proving the program- . 
mes and organizations involved in development at the base, and not those who are 
intending to control it from the top. Ina way, Participatory Evaluation is an attempt 
at redefining and reaffirming development as a “bottom-up”, “people-centered”, 
“people-controlled” process and not a technocratic, top-down intervention. It is _ 


this thrust that provides the distinctive meaning to Participatory Evaluation 
methodology. . | oe 


Participatory Evaluation is to 
be seen as a process of 
individual and collective 
learning. In a sense, it is an 
educational experience. 


WHATIS 


PARTICIPATORY 
EVALUATION? 


Before we answer the question what is Participatory Evaluation, it may be 
useful, first of all to become clear as to what is evaluation. The literal 
meaning of the word ‘evaluation’ is ‘‘to determine the value of’’. This 
essentially means to assess the worth of, the value of, a given activity. This 
implies at least two things: first, that a given activity is to be assessed ina 
given context — the process of assessment, of analysis, of review is critical 
in any evaluation; second is to assess the activity in relation to something 
and that something is the goal or the objective of the activity. All develop- 
ment programmes and plans evolve with a giveri set of objectives and 
goals. We evolve these goals and objectives on the basis of certain broader 
understanding and values. In order to achieve these goals, certain ac- 
tivities are planned. These plans are then intended to accomplish the 
above goals. Once the plans have been implemented, we can then assess 
whether these activities really led to the accomplishment of these goals or 
not. This is the process of evaluation. (See Chart I). 


Participatory Evaluation implies that it is developmental and is in the 
interests of those who are involved in planning and implementing the 
given developmental activities and benefitting from the same. Thus, the 
process of evaluation is to be controlled by those whose activities, initia- 
tives, plans and outcomes are being evaluated. Another characteristic of 
Participatory Evaluation, given its above concern, is to ensure that it is a 
collective process of reflection, planning and control and that it is not a 
process whereby a single individual reflects on behalf of others and pre- 
sents outcomes of his/her reflection for others to use in developing their 
plans. This collective nexus of Participatory Evaluation ensures a wider 
control as well as broader developmental possibilities. 


Finally, Participatory Evaluation is to be seen as a process of individual 
and collective learning. In a sense, it is an educational experience. It is 
learning about one’s strengths, about one’s weaknesses; learning about 
the way plans and programmes get implemented; learning about social 
processes and development outcomes; learning about social reality and 
intervening in the same; learning about creation and development of 
organizations and ensuring theit relevance and longevity. It implies 
clarifying and rearticulating one’s vision and perspective about the develop- 
ment work we are involved in. This educational thrust of Participatory 
Evaluation methodology implies that various parties involved in a develop- 
ment programme experience Participatory Evaluation as a learning pro- 
cess for themselves. And, the process is designed and structured in such 
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The Participatory Evaluation 
methodology is rooted in a 
certain world-view, a certain 
vision about human beings 
and their capacity and, 

' therefore, in a certain 


interpretation of social reality. 


arning. It is not merely the outcome of Par- 
ticipatory Evaluation which provides insights and learning, it is also the 
very involvement in the process of Participatory Evaluation that becomes 
the basis for learning and education. Thus it creates conditions conducive 
to,and willingness for, change for action. This is a crucial distinction 
between Participatory Evaluation and _ conventional eyaluation 


methodologies. 


a way that it ensures that le 


At the core of Participatory Evaluation methodology, therefore, is our 
faith in ordinary people, grass-roots workers, community organizers, 
adult educators etc., our belief that they are themselves interested in 
improving their practice, sharpening their vision and developing them- 
selves; and that they would be interested in, and committed to, evaluating 
themselves, their activities and programmes in order to do so. It is this 
faith in ordinary people, in their willingness and capacity to get involved 
in a critical reflection exercise that provides the philosophical underpin- 
ning to Participatory Evaluation methodology. This is the basic difference 
between looking at Participatory Evaluation as a developmental experi- 
ence, an educational experience as opposed to a regulatory mechanism, of 
control over people, programmes and resources. 


It is, therefore, important to recognize that Participatory Evaluation 
methodology is rooted in a certain world-view, a certain vision about | 
human beings and their capacity and, therefore, in a certain interpretation 
of social reality. It is not a mere tool or technique which is distinct from 
other tools or techniques and can be mechanically applied and im- 
plenvented in any context. Its historical and contextual underpinning lies 
in the process of making development, and related activities, controlled 
by the people at the base in their own collective interest, and it provides 
the basic meaning to Participatory Evaluation methodology. Without that 
context, it may be misappropriated and misinterpreted, as a mere set of 
tools and techniques. 
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The Participatory Evaluation 
methodology is consistent 
with participatory models of 
development which are now 
being experimented with at 
the grass-roots level 
throughout the world. 


WHy 


PARTICIPATORY 
EVALUATION? 


Having clarified what is Participatory Evaluation methodology, it may be 
useful to spend some time understanding its relevance and importance in 
the context of development. It is our view that Participatory Evaluation 
methodology is consistent with participatory models of development 
which are now being experimented with at the grass-roots level through- 
out the world. In the néw models of development, people are at the 
centre, and are involved in creating their own plans and programmes for 
their development, and such plans and programmes are implemented 
through their active involvement and overall control. Given this thrust of 
people-centered, bottom-up development, it makes consistent sense to 
ensure that the process of reflection and evaluation has similar charac- 
teristics and meaning. It will be a contradiction to have a people-centered, 
bottom-up process of development evaluated through commissioned 
agents appointed by resource-providers. It is in this context that the value 
of Participatory Evaluation becomes heightened. 


In the context of such a development initiative specifically, Participatory 
Evaluation has been utilized with three broad emphases. (See Chart II). 
The first emphasis of Participatory Evaluation has been on the program- 
mes and activities within a given development context. The assumptions 
behind developing a programme are tested through a Participatory Evalu- 
ation exercise after the programme has been implemented for a period of 
time. These programmes and activities could be an adult education effort, 
an income-generating programme, a rural development initiative or a 
health care programme. Development programmes are planned to ac- 
complish some short-term goals, and evaluation of the activities at the 
programme level can help us to assess whether those goals have been 
accomplished and to what extent, and what came in the way of doing the 
same. Thus, Participatory Evaluation has been largely utilized in assessing 
the impact of a given programme, in assessing the underlying assump- 
tions by which those programmes were created, in assessing the relevance 
of those programmes in the context of changing social realities, and in 
assessing the manner in which those programmes were implemented. 


Thus, the focus of such an evaluation exercise is essentially on the 
field’ and it largely entails active involvement of local population, people 
who are likely to benefit and gain from the programmes and field-level 
staff and organizers. This is a very important emphasis in Participatory 
Evaluation because it helps us to improve our programmes in the future 
and to strengthen the possibilities of accomplishing those goals and objec- 
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The focus of an evaluation 
exercise is essentially on the 
“field” and it largely entails 
active involvement of local 
population, people who are 
likely to benefit and gain from 
the programmes and 
field-level staff and 
organizers. 


n evaluation also helps us to define new prog- 


tives in the future. Such a plementing 


ramme thrusts, new programme activitiés, new ways of im pr 
programmes and activities, and extending the dimensions 0 g 


programmes. 


The second major emphasis in Participatory Evaluation has ne oa 
development-promoting organization itself. In order to plan, impleme 
and sustain development programmes and activities in a given — rs 
all create organizations as mechanisms for the same. Over a perio ; 
time, these organizations have to develop their own internal capacity to 
continue this process of development. As people join in these organiza- 
tions, as systems, procedures and structures get created in these organiza- 
tions, as the external environment begins to interact with these organiza- 
tions, various organizational issues begin to emerge in the history of all 
development-promoting organizations. These issues relate to 
people-to-people interaction, interpersonal relations, issues of 
team-building, conflict between young and old, new, members and old 
members, field staff vs head office staff, etc. 


Organizational issues also include the tension between the need for 
routine structure and spontaneity and flexibility, between the need to 
create a minimum agreed common ground and the need to provide space 
for individual creativity, the need to ensure a common understanding of 
the organization’s missions and goals and tasks, and the need to provide 
opportunity for members of the organization to continuously redefine 
those missions, goals and tasks. Over a period of timé, all develop- 
ment-promoting organizations begin to interact with their social, political 
and regulatory environment. Various laws and rules governing them 
begin to affect them; donors and other resource-providers begin to influ- 
ence them and the organizations need to evolve strategies to not only 
cope with these segments of the environments, but also to influence them 
more actively. Over a period of time, all development. promoting organi- 
zations, as a consequence of their success, begin to expand and grow into 
new areas, new programmes, new activities and new staff. Growth brings 
new issues in its wake and organizations begin to face a series of issues 
arising out of this growth process. These and many more such organiza- 
tional issues become a necessary focus of attention in the context of 
development. 


A Participatory Evaluation exercise, with its emphasis on the organiza- 
tion, can help us reflect on these issues, and others, and to evolve 
strategies to deal with them in an open, critical and collective fashion. A 
Participatory Evaluation intervention around such issues can help bring 
about a common and shared understanding of the problems and collective 
efforts to solve them. Such a Participatory Evaluation intervention can 
help in developing the organization and ensuring its smooth, strong and 
dynamic future. Such a Participatory Evaluation intervention with em- 
phasis on organizational issues, has a process that is “‘institution-focus- 
sed” and entails the active involvement of the field staff, senior members 
of the organization including its governing body members and other key 
parties in its environment. The need to utilize Participatory Evaluation 
intervention around such organizational issues is only beginning to be 


A Participatory Evaluation 
intervention with emphasis 
' on organizational issues, has a 
process that is 
“institution-focussed” and 
entails the active involvement 
of the field staff, senior 
members of the organization 
including its governing body 
members and other key 
parties in its environment. 


A Participatory Evaluation 
intervention emphasizing a 
reflection of the perspective is 
“vision-focussed” and it 
necessarily entails active 
involvement of senior 
members of the organization 
and all those who are involved 
in it, as the vision and 
commitment of all 
development-promoting 
individuals becomes the focus 
of the intervention. 


recognized among the development community in recent years, Our 
experience has suggested that this is an extremely crucial area where a 
Participatory Evaluation exercise can be of immense value in ensuring 
strength, dynamism and vitality of a development promoting organiza- 
tion. 


The third area where Participatory Evaluation has put emphasis is on 
the perspective with which a development initiative is undertaken. De- 
velopment is a socio-political issue and it entails a certain vision and 
commitment. We all have a vision about what society ought to be and a 
commitment for changing the given existing system towards that vision 
Our perspective helps us determine how this change process will occur 
and what could be our role in supporting and strengthening such a 
change process. All development programmes are created as a consequ- 
ence of this perspective. Our perspectives are based on certain assump- 
tions which can be refined through Participatory Evaluation. A Participat- 
ory Evaluation exercise can help us redefine our perspective in the context 
of changing social reality. 


Many times the social reality around us changes so rapidly that we need 
fo re-examine our perspective and redefine our role in the context of the 
new reality. A Participatory Evaluation exercise can also help us to reaf- 
firm our vision, our faith and our commitment. It can help us to sharpen 
our understanding of the social reality and possibilities of our intervention 
in the same. A reflection process catalyzed through Participatory Evalua- 
tion with emphasis on perspective may entail a rejuvenation of our com- 
mitment, our vision, our direction, our understanding, our ideology, and 
the potential possibilities of our role in the future. Many a times develop- 
ment-promoting individuals and organizations experience a certain sense 
of stagnation, a plateau in their work and become confused about which 
way to go, what are the future steps and directions in their own role. It is 
in these moments we have found that a Participatory Evaluation interven- 
tion has been extremely useful in not only reclarifying but also reaffirming 
the vision, commitment and perspective. 


Thus, a Participatory Evaluation intervention emphasizing a reflection 
of the perspective is “vision-focussed” and it necessarily entails active 
involvement of senior members of the organization and all those who are 
involved in it, as the vision and commitment of all. development-promot- 
ing individuals becomes the focus of the intervention. 


Thus, a Participatory Evaluation intervention in a development context 
can be utilized to reflect on the programme, or the organization or the 
perspective. In reality, all these three are intertwined. And, many a times, 
Participatory Evaluation exercise helps identify congruence, or otherwise, 
between perspective, programme and organization. This can become a 
major outcome of such an exercise. We have found that the primary 
emphasis in context to above three areas of a given Participatory Evalua- 
tion intervention determines the scope, the nature,.the timing and the 
methods utilized, as can be seen from Chart II. 


In the case-studies presented, three (Village Development Trust, Char- 
ity Bengal and Andhra Pradesh Balwadi Programme) are based on the 
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primary emphasis on programmes, the other three (Inter School Project, 
Jagriti and Rural Development Organization) have a primary emphasis on 
their own organization; and the last two (Workers’ Education Project and 
Tribal Development Society) have a primary emphasis on the perspective. 
As the case-studies themselves show, other aspects also get involved in it, 
though the primary emphasis continues to determine the scope and the 
depth of the evaluation. 


based on our faith that people are interested in improving their 
practice and sharpening their vision; | 
developmental in nature; and in the interest of those who are 
ie actual actors of the activities to be evaluated; 
ee “process controlled by those whose activities, initiatives, plans- 


Participatory Evaluation is 


: | and O itcomes are evaluated; 
a collective process of reflection and planning; 
ducational experience for those involved in it. 


ORGANIZATIONAL GOALS 


PROGRAMME OBJECTIVES 
IMPLEMENTING STRUCTURE 
FIELD IMPLEMENTATION 
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Participants Local people Field staff Senior members of 
the organization 
Field staff Senior members of 
the organization 
Governing body 
members 


Discussions, 


Perceptual data, 
meetings, etc. 


interviews, group 
meetings 


Questionnaires, 
interviews, records 


TRIBAL DEVELOPMENT 


VILLAGE DEVELOPMENT RURAL DEVELOPMENT 
TRUST ORGANIZATION SOCIETY 
CHARITY BENGAL JAGRITI WorKERS’ EDUCATION 


ANDHRA PRADESH INTER SCHOOL PROJECT PROJECT 


BALWADI PROGRAMME 


Cuart II 
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Howls A 
PARTICIPATORY 
EVALUATION 
EXERCISE 
PLANNED AND 
(CONDUCTED? 


Much of this becomes clearer in the context of the case-studies presented 
in the next section. We believe that there is no readymade formula which 
can be applied in all situations. What we can suggest are a series of steps 
that we have used in our own context. Some of the key steps in the process 
are Outlined in this section. 


1. SETTING OBJECTIVES: FRAMES OF REFERENCE 


The first task in any Participatory Evaluation exercise is to set objectives of 
the evaluation. We have traditionally utilized an initial workshop, with all 
key actors in the development situation, to set the objectives of the evalu- 
ation. We find that this is the most crucial step where discussions on why 
we need evaluation at this stage, to whom it is going to benefit, what problems 
it is going to cause to our work and what should be the key objectives of 
such an evaluation exercise are discussed at length and sorted out. At the 
end of this step the focus of evaluation gets determined and articulated. It 
is here that the distinction between the emphasis on programme, organi- 
zation or perspective begins to get stated. This helps us set the frame of 
reference within which a given evaluation exercise then proceeds. 


2. IDENTIFYING PARAMETERS AND 
INFORMATION NEEDED 


Having agreed upon the objectives of evaluation, the next step is to 
identify what kind of information is needed on what parameters. For 
example, if the purpose of the evaluation is to assess the impact of the health 
programme, then the infant mortality rate could be one of the parameters 
on which we need to get information. This is an exercise that requires 


some reflection on what is it that we are trying to assess. It is the statement 
of those dimensions, those factors, those aspects, those variables, which 
we are trying to assess that this step entails and it may involve a certain 
degree of reflection and thinking before these parameters get articulated. 


3. IDENTIFYING SOURCES OF INFORMATION 


Once we Know what information we want, then we can identify where we 
are going to get it from. The source of information could be members in 
the community, field staff, senior members of the organization, parties in 
the environment (like government officials), records, reports, documents, 
actual field situation etc. Different types of information can be obtained 
from different sources. Where people’s impressions, views, experiences, 
perceptions, ideas, feelings are important, people themselves become the 
major source of information. Besides, historical and archival records and 
documents can also provide valuable information. 


4. DEVELOPING METHODS TO OBTAIN THAT 
INFORMATION AND DATA COLLECTION 


Once we know what information we want and what is the source of that 
particular information, we can then develop methods to obtain that infor- 
mation. Methods of data-collection can be varied: we can have 
open-ended interviews, questionnaires, structured interviews, group dis- 
cussions, field observations, study of records, physical measurements, 
etc. In addition to the use of all these methods which have also been used 
in traditional evaluation practices, newer methods of data-collection can 
also be evolved. Folk media, songs, role-plays, dramas, art, drawings, 
etc., have also been extensively used in getting information through 
popular means. Essentially this step entails developing appropriate 
methods for collection of information and then collecting that informa- 
tion. 


5. ANALYSING DATA 


Once information has been collected from different sources in different 
ways then it has to be put together, consolidated, categorized and 
analysed. Some common patterns, some variations, some links, some 
relations, some measures have to be developed through this process of 
analysis. It is important that at this stage of analysis a collective process is 
supported and encouraged, so that analysis becomes a shared one, as 
opposed to an individual one. The point needs to be emphasized here 
that the collective analysis thus reached should be disseminated to all 
those constituencies from where this information has been taken. This 
“feedback” is an important step in the process. 
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6. CREATING FUTURE SCENARIOS 


Inevitably, we have found that analysis of the past is very closely linked 
to creating future scenarios. In our practice of Participatory Evaluation, 
we have deliberately utilized this step to assist parties involved in the 
evaluation process to paint desirable future scenarios on the basis of the 
analysis so developed. Sometimes this step also entails assessing the 
outside environment in order to paint such future scenarios for their 
work, for their programmes, for their local situations, etc. 


7. EVOLVING ACTION PLANS 


Multiple future scenarios are then discussed, sorted out and integrated 
and the parties are encouraged to develop concrete action plans arising 
out of the analysis and the agreed upon future scenarios. These action 
plans, at least in the context of Participatory Evaluation, are broadly 
identified, and not concretely planned in detail. That could be a step 
following the evaluation exercise. 3 
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‘SHARINGRESPONSIBILITIES 
_ The crucial step cutting through all the . steps of the planning _ 
"process is to ensure that responsibility for these various tasks in the evalua- _ 

tion process is shared between different parties —local people, field staff, — 
_ other members of the organization, facilitators of the evaluation, etc. This is — 
important to ensure authenticity in data-collection and rigour in analysis, as _ 
_ well as in sustaining the ownership of diferent paris to the evaluation — 
_ process. A time-plan is also developed at the same gee eae 
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We now invite you to study the case-studies in the next section. These 
illustrations will help clarify some practical questions in Participatory 
Evaluation, they will also provoke some issues for reflection. The last 
section addresses some of those issues. 


VILLAGE 
DEVELOPMENT TRUST 


The Village Development Trust (VDT) has been working for the last 12 
years in coastal Tamil Nadu with the rural poor. It started its work in the 
mid-seventies, in the wake of the severe cyclone which washed away a 
number of villages and rendered people homeless and jobless. VDT used 
its Own resources and that of the government to help rehabilitate cyc- 
lone-affected people, helping people build their houses, providing them 
agricultural inputs for developing and reclaiming the land. Over the years, 
Village Development Trust has also undertaken a variety of other prog- 
rammes like animal husbandry, health care, adult education, social fores- 


try, etc. 


VDT began working with the women in the villages encouraging them 
to save a small amount of money every month. Over a period of time, 
these Savings Societies began to offer loans to their members for small 
economic activities like buying of a goat or a hen or seeds, etc. Later, VDT 
bought its own resources also and made them available to these savings 
and thrift societies of the women in the villages. Thus larger amounts of 
funds became available for women to use in different economic program- 
mes. Most of these were given to individual women as loans to pursue an 
economic activity. Milch animals became a major programme and VDT 
developed veterinary care and training of local, illiterate, and semi-literate 
women as bare-foot veterinarians. After a few years of functioning of 
these savings goups, which began to be called Mahila Mandals, in 32 
villages, VDT helved create an umbrella organization which was intended 
to be the overall guiding institution of various individual village-based 
Mahila Mandals. 


This evaluation process was initiated at the joint request of VDT and the 
donor agency supporting its women’s programme. It was felt that since 
VDT was considering expansion of the programme to another 25-30 vil- 
lages, it would like to take stock of its existing programmes before it 
moved forward. An initial discussion with VDT representatives, donor 
representatives and representatives from the intervening agency took 
place. This was followed by the visit of a four-person team to the area of 
work of VDT. After a brief survey of the field and an initial familiarization 
with the programme, it was decided to plan the evaluation along with the 
field and programme staff working on the women’s programme in VDT. 


This planning focussed upon looking at various aspects of the function- 
ing of Mahila Mandals, the nature of membership and leadership in these, 
the kinds of activities undertaken by them and the future potential of 
these women’s organizations at the village and local levels. The first step 
in the planning was to have a day long workshop with the ten field staff 
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involved in the women’s programme of VDT. The facilitators encouraged 
reflection of the staff to identify important characteristics of what they 
conceived as strong Mahila Mandals. A series of characteristics were 
listed and consolidated after discussion. Each staff member was then 
requested to evaluate the existing 32 Mahila Mandals on these characteris- 
tics and to classify them into three types: strong, average and weak. Out 
of this classification, 15 Mahila Mandals were chosen for indepth field 
visit by the intervening agency and VDT teams. Out of these selected 
Mahila Mandals, five were strong, five average and five weak. 


A common framework for collection of information in the field was 
evolved by the intervening agency and VDT and guidelines for interviews 
with members and leaders of Mahila Mandals and criteria for study of 
their records were jointly elaborated so that a comparative analysis across 
Mahila Mandals could be undertaken. 


Over the next three months, indepth field visits were made in some 
Mahila Mandals by a joint intervening agency-VDT team, in some others 
only by the intervening agency team and in some villages only by VDT. 
These field visits entailed discussions with leaders of Mahila Mandals and 
members, observation of their monthly and weekly meetings, study of the 
records kept by these at the village level and discussions with the field 
staff working with those specific Mahila Mandals in the field. Simultane- 
ously, indepth interviews with other staff involved in the VDT prog- 
ramme were also held to understand the history, orientation and impact 
of the programme from their point of view. 


Based on the data-collection in the field, reports on each Mahila Mandal 
were prepared by the intervening agency and VDT. These reports were 
then shared and discussed in a workshop with senior VDT staff in the 
field. On the basis of these individual reports on Mahila Mandals visited, 
an overall draft report with key findings was prepared. These key findings 
were then further discussed in the workshop with four senior VDT staff 
along with the intervening agency team and donor representatives, and 
modifications were made thereto on the basis of additional information 
and analysis. Some of the key findings from the evaluation are presented 
here. : 


KEY FINDINGS 
The following are some of the key findings based on the above: 
ENTRY 


1. Cyclone relief and rehabilitation efforts provided initial entry and 
credibility to VDT in the villages of the area. As a result of its land 
reclamation and agro-input programmes, VDT made a-credible entry to 
the villages in 1978. This was further reinforced in some other places 
following the 1983 cyclone. Loans arranged through banks for purchase of 
bullock-carts, artisan implements and milch animals helped considerably 
in rehabilitating affected families. This Provided opportunity for VDT to 
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gain entry to, and credibility with, the local village and provided the 
foundation for its future work. It was this involvement of VDT that also 


later led to substantial infrastructural improvement in the area— roads, 
electricity, bank branches, etc. 


2. Thrift collection provided the basis for bringing women together 
and organizing Mahila Mandals. In early eighties, VDT began to encour- 
age poor rural women to save, and pool their savings in a thrift account. 
These savings then were used to advance small loans. In fact, women in 
four villages started collecting thrift in 1979 on their own, and VDT started 
promoting it in 1981. The main thrift and loans programme started in 1982 
and spread in 1983. This thrift collection activity has brought women 
together in 32 villages. Now there are about 4000 women members in 
these thrift societies and savings total about two lakhs rupees. 


MaAnHILA MANDALS AND THEIR LEADERSHIP 


3. The Mahila Mandals gradually generated women leaders who 
began to operate these units. These thrift collection units were called 
Mahila Mandals (MMs). Initially, each MM had a secretary and a presi- 
dent. This was discontinued three years ago as some of these leaders 
began to develop vested interests. Now, there are Advisory Committees 
(AC) for each MM. The AC meets regularly (fortnightly or so) to review 
loan applications and monitor repayments. Membership in AC varies 
from three to 12, depending partly on the size of MM. Yet, in many 
villages (10 out of 15 visited) old presidents and secretaries continue to be 
members of AC and are recognized as such. The MMs became operating 
units for thrift, and AC became the leadership group of MM. It is clear that 
many of the AC members have emerged as important and confident 
women leaders in those villages. 


4, AC leadership is largely representative of MM membership. The 
leadership represented in AC, by and large, represents the membership. 
One third of the membership of MM comprises better-off rural women 
(criteria like land ownership of three acres or above, pucca house, not 
doing wage labour, etc.); the rest are marginal farmers (less than one acre 
of land), landless labourers, potters, fishermen, etc. The leadership of AC 
is slightly more than half in the hands of better-off women. Most women 
members of AC, at least the active ones, come from somewhat better-off 
families. It seems that VDT first concentrated on getting the women out, 
any woman, and were not bothered about poor women only. The compos- 
ition of MMs and ACs shows no class uniformity. 


5. Key women leaders in most Mahila Mandals are economically bet- 
ter-off. Though each AC has some landless and harijan and wage labour 
women as members, the key leadership is mostly exercised by asomewhat 
better-off woman. In one of the villages the secretary’s family owns eight 
acres of land; in another village, the president is considerably well-off. 
Wage iabourers and other poor women do not attend most of the AC 
meetings since these are held in the afternoons. On an average, atten- 
dance of AC members is 60 per cent. In one village, harijans are effectively 


COMMUNITY HEALTH CEL 21 
(NZ S 24 326, V Main, | Block 
; Koramengala 
CON H "oy ee Bangalore-560034 er 


India 


A 


22 


excluded from MM; in another village, most landless women do not show 
any interest. At the same time, three-four landless women have also 
begun to exercise considerable confident leadership in some MMs. How- 
ever, heterogenous membership of MMs and ACs tends to favour bet- 
ter-off women taking over key leadership functions. 


6. Collective and strong women’s leadership has not emerged in most 
MMs. While individual women with confidence, courage and compe- 
tence have emerged as leaders in most MMs, building of strong collective 
and shared leadership of MMs has not taken place. There have been many 
examples of infighting, splits, absenteeism, interpersonal conflicts and 
individualized functioning in some MMs. There is continual conflict bet- 
ween the president and the secretary of one MM; faction leaders dominate 
another MM; women’s leadership in another MM has witnessed a split; 
one woman alone leads in another MM. In the course of the functioning 
of AC and MMM, several tensions and difficulties have emerged which 
have hampered the growth of strong and collective women’s leadership. 
It needs to be recognized that these tensions and conflicts are inherent in 
this type of effort; but focussed handling of these processes can help 
overcome them and thereby contribute to a shared and collective leader- 
ship in these MMs. 


7. AC leadership is perceived to have cornered most of the benefits 
from loans and programmes. The data analysed on loan as for the last 
three years does not suggest that many loans have been cornered by AC 
members. Two things are important here. First, some initial loans made 
available through the umbrella organization have first gone to AC mem- 
bers themselves. This seems to be particularly so in one village (nurseries 
with AC members, six paddy loans to AC members). Second, ordinary 
members perceive AC members favouring themselves and promoting 
favouritism. The AC leaders have been accused of being selfish and 
self-centered three villages. In some places, women feel that AC leader- 
ship intentionally supports loan applications of only one caste, and not of 


others. One AC member in one village got nine loans without repaying 
earlier ones. | 


The important thing is that leadership has not been sensitive to issues 
of equity and perceived equity. Ordinary members may lose interest in 
MM if AC leadership is perceived as self-supporting. 


8. Mahila Mandals have become largely restricted to thrift societies. 
Since thrift brought the women together, it remained an important activity 
of the MMs. However, this thrift has become the sole purpose of many 
MMs. AC has become the leadership group, both in name and deed. All 
AC meetings, without a single exception, talk about thrifts, loans and 
repayments. Most monthly meetings of MMs only discuss loans, repay- 
ments and savings. As a consequence, only those interested in loans 
attend AC or MM meetings; average attendance at MM meetings is less 
than 20%; most monthly meetings of MMs are not held if VDT senior staff 
do not come as women say they have nothing to discuss; some women 
who have outstanding loans in their name shy away from AC meetings 
and MM meetings as they do not want to be questioned; problems of 
women and the village as a whole do not get much attention inside or 
outside the meetings. 


Several villages have serious problems of drinking water, lack of elec- 
tricity, fodder, alcoholism, etc. Though a few MMs have taken some 
initiative in this regard, this is not a common practice. Some references to 
these problems have been made in some AC or MM meetings, but no 
effective discussion or action planning followed. This does not deny some 
examples of women’s:actions on other issues of common concern in the 
village. The point-is that: MMs, by and large, seem to have focussed on 
problems of thrift, loans and repayments only. 


The confusion between MM membership and that of thrift society also 
exists. This confusion confounds the boundary between MM and thrift 
further. And it does not become clear to a;woman who is not involved in 
thrift if she has a place in MM or not. In fact, MMs, as per their activities, 
seem to be merely thrift promoting and loan giving mechanisms. 


9. Women’s Umbrella Organization is a wing of VDT and is not per- 
ceived as a women’s organization. Women’s Umbrella Organization 
(WUO) has been mentioned as a women’s organization, though the way 
it is structured and operated, it is a wing of VDT. It is not an umbrella 
organization of all MMs, nor is seen as the women’s own organization. 
Most women interviewed in the field, including a majority of AC mem- 
bers, associated the loans being given as VDT loans. They did not mention 
that these loans were being given by WUO, although VDT had planned 
and hoped that the loans would be seen as WUO loans and not VDT 
loans. WUO appears to be a,conduit for funds and programmes from 
VDT, not a larger organization of MMs. Even reference to WUO in some 
meetings appears to be made. as VDT operation, and not something 
created and owned by women and Mahila Mandals themselves. The 
question is, what is and what should be the identity of Women’s Umbrella 
Organization? Clarity on this will help in furthering the organization. 


As the follow-up of this, VDT was then requested to plan the next phase 
of the programme in the light of these findings, including plans for de- 
velopment and strengthening of its field staff to undertake this prog- 
ramme. The intervening agency was requested to assist VDT in this effort, 
both by VDT and the donor, which it agreed to do as a follow-up. The 
entire exercise was spread over a period of eight months and entailed five 
rounds of field visits by the intervening agency team. 
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ISSUES 


Some of the main issues emerging from this experience are elaborated 
here: 


1. When the initial proposal for evaluation was mooted, VDT seemed in 
a great hurry to expand its women’s programme to an additional 25 
villages. The overall time-frame set for the evaluation, initially spanning 
only four months, was conditioned by this consideration. However, the 
experience of evaluation, in particular some difficult and penetrating 
questions raised by the intervening team, seemed to delay the expansion 
plans. In fact, the follow-up planning as a consequence of evaluation did 
not proceed as planned, and the intervening agency team was not in- 
volved. The initial enthusiasm of VDT seemed to have evaporated by the 
end of the process. 


2. The second issue relates to the coordination of evaluation within VDT. 

Over the duration of the evaluation, coordination of this exercise in VDT 
changed hands four times. A woman was initially named by VDT as the 
key contact for the intervening team. In the second visit, the team disco- 
vered that this woman had gone on long leave and a man was now 
nominated to coordinate the evaluation exercise. During the third visit, 
the team was told that a third person was coordinating the evaluation 
since the man had gone away for some training. And finally, the head of 
VDT himself took over the coordination. This sequence made the task of 
the intervening agency rather difficult, as each new coordinator did not 
fully know what the previous one had done. Besides, it also reflected the 
rather disorganized state of affairs in VDT itself, and the seriousness with 
which they got involved in the evaluation exercise. 


CHARITY BENGAL 


CONTEXT 


Charity Bengal (CB) is a voluntary agency working in West Bengal for the 
past 18 years. It has helped small farmers with irrigation and new forms of 
agriculture over the years. Gradually, it began to realize the importance of 
educational efforts with the rural poor as an essential component of their , 
programme to improve their situation. The CB project was, therefore, 
planned initially for a three year period (1979-81) as an educational effort 
aiming at “empowering people by raising their consciousness level, 
mobilizing them towards movements, and building up those systems and 
institutions necessary to ensure that the producer enjoys the fruits of his 
labour.” Since then, the project has been extended for another five years 
(1982-86) and is currently being implemented in 21 Development Zones of 
six regions. 


Specifically, the objectives of the CB project are stated as follows: 


(i) To train personnel in order to identify, select and train leaders of the 
partner groups for action programmes. 


(ii) To identify and organize groupwise the exploited sections of society 
through their involvement and participation in various activities. 


(iii) To involve and organize women’s groups for participation in the 
development process of their families and of the society itself. 


(iv) To help form associations of various partner groups for solidarity and 
joint activities. 
The project is being implemented in 448 villages in 14 districts. Its major 


work strategy has been the formation of partner groups (PGs) of landless, 
marginal farmers, women, youth, etc. 


PROCESS 


Though the initial request for evaluation came from the donor agency, the 
exercise for this evaluation was carried out in such a manner that the 
project staff and members of PGs would experience it as a developmental 
exercise rather than a judgmental one. In order to facilitate this process of 
participatory evaluation, the CB staff was requested to formulate its own 
objectives of evaluation, identify priority areas for focus during field visits, 
select regions, zones and PGs for field visits to ensure a wide range of 
coverage, and suggest methods of evaluation to ensure involvement of 
PG members in this exercise. The response of CB staff as well as funders’ 
suggestions, combined to generate the following objectives of evaluation: 
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(a) the impact of the project on self-reliance of PGs; 
(b) successes and shortcomings of the project to date; 
(c) organizational viability and motivation of PGs. 


The central method of evaluation was through workshops, group dis- 
cussions and self-assessments. Besides, previous records, reports and 
documents were also perused. Most of the statistics were provided by 
project staff, though some, like membership, background information, 
was collected during the field visits. The project staff had proposed field 
visits to three regions. These represented three distinct geographical and 
socio-economic areas and also had variations in the extent of project 
implementation. In all, 18 PGs (and their members) were met and discus- 
sions held with them. 


Preliminary findings from the field units were shared with the senior 
staff at the end of the visits, first in each region, and then in the central 
office. The discussions, analysis and insights from there were incorpo- 
rated in the draft of the report prepared by the evaluation team. The CB 
staff used this draft for further discussions amongst themselves and for 
preparing future action plans. The intervening agency was not involved 
in this effort. 


SALIENT FINDINGS 
1. - The organization of partner groups 


One of the main tasks under the CB project is to organize people into 
viable groups which can then initiate their own development. The total 
number of partner groups (PGs), as these are called, stood at 1028, with a 
membership of 20,875. Thus, in sheer size and quantity of membership 
and PGs, this appears to be a considerable effort. The PGs are of several 
varieties: male landless, marginal farmers, women, youth, children, etc. 


2. Achievements 


Several significant accomplishments of PGs have been assessed. The 
major ones are listed below: 


(a) Increase in social awareness appears to.be an important one. 
Though variations across membership exist, it is clearly evident that 
members of PGs have become more aware of their situation, need 
for unity and solidarity, and taking collective initiatives for their 
own development. Awareness of prevalence of superstitions and 
socially unjust practices has also increased among members. 


(b) Accumulation of savings by different PGs is another landmark 
accomplishment of the project. Total accumylated savings currently 
stood at Rs 28,63,349. Though variations exist across PGs on the rate 
and volume of savings, it is clear that regular savings have become 
an integral part of PGs. All members seem to have regularly filled 
pass books, and weekly/fortnightly PG meetings are occasions for 
collection of savings. The rate of Savings varies from Re 0.50 to 


Rs 5.00 per week. Women and youth PGs have also adopted systems 
of regular savings. 


Two important outcomes of this practice of savings have been 
decreased dependence on outside moneylenders and use of their 
Own resources. Many members belonging to PGs reported getting 
out of the dependence on local moneylenders, as they now get 
loans from the savings in PGs. Examples of PGs undertaking small 
income-generating activities, through partial or full use of theirown 
savings, also abound. 


(c) Acquisition of skills and knowledge, particularly in areas of agricul- 
ture, pisciculture, poultry, goat-rearing, primary health care, etc., 
has been another major gain for members of PGs. Obviously there 
are variations across PGs on the knowledge and skills they have 
acquired and used, as farmer groups seem to have benefited most 
consistently as compared to landless groups. Learning to plan and 
implement their own collective activities was also occasionally visi- 
ble in a few PGs, and almost all members have learnt how to sign 
their names. 


(d) Collective actions against social injustices have been reported in 
some cases. The Nischintapur landless group fought against a pow- 
erful man’s encroachment on their common land and demonstrated 
in front of the DM’s house; the members of several landless groups 
in another zone struck work for three days to demand a better share 
in share-cropping arrangements; the members of Pekua village have 
protested against bank employees taking a cut against the loans 
granted"by them. These examples are only illustrative and indicate 
the increasing trend of collective actions by PGs. 


3. Assessment of partner groups 


The variations across PGs in the extent of their organizational viability 
clearly exist. Field workers in the three regions visited by the evaluation 
team themselves assessed these PGs. These results indicate that a little 
more than one-third of the PGs have achieved a good viability. While 
short duration of existence was one of the main reasons given for weak 
groups, many of them have also existed for about two years. 


A variety of assessment criteria were used by field-workers in making 
these assessments of PGs. These are identical to many elements of criteria 
evolved by senior staff of CB at the start of this evaluation. This set of 
criteria was used by field workers in each of the three regions to assess one 
PG each. This suggests the feasibility and desirability of this exercise. 


4. Issues for consideration 


The following main issues were posed for consideration and appropriate 
action regarding increased organizational viability of PGs: 


(a) Diffuse criteria for membership in PGs 


There are, broadly speaking, four types of PGs under the CB project: 
landless male, marginal farmers, women and youth. Besides, some 
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children’s groups and mixed groupsalso exist.The criteria for mem- 
bership in these categories of groups are somewhat diffuse. In one 
zone it was stated that members of landless (male) PGs owning land 
less than 0.50 acre (including homestead land) were considered and 
this was strictly adhered to. The land ownership pattern of 16 mem- 
bers present on the meeting of Para Landless Group confirmed this 
fact. In another zone, the criterion for defining landless was stated 
as 0.40 acres, but in the meeting of Nazirpara Landless Groyp two 
members (including the chairman of the PG) had more land than 
0.40 acres. The criteria for defining membership of landless group 

in the third region were not stated clearly and precisely. | 


The criteria for membership of the marginal farmer groups are 
much more diffuse. Though called marginal and the limit set at 
three acres, it is not being adhered to in any of the zones visited. 
This is so even by the admission of field staff. 


The situation with respect to youth groups is perhaps worst in this 
regard. The Jubo Sangh in the first zone is practically controlled by 
young men of two rich families of the village; it has two share-crop- 
pers and one landless man as members; one young boy, a member, 
is the son of another member. 


The focus of the project will be diffused unless precise criteria for 
membership to PGs are defined and strictly adhered to. Otherwise 
the PGs are not homogeneous, with membership of common in- 
terest. 


Socially unjust practices in PGs 


The PGs have evolved certain practices within their functioning 
which perpetuate existing socially unjust practices prevalent in the 
area. Some illustrations are given here. The rates of interest on loans 
extended to members from the savings fund of the PGs, though 
lower than prevalent private moneylending rates, are still often 
exorbitant. The. rates of interest charged by PGs vary from 40% to 
even 100% per annum. 


Similarly, unjust share-cropping arrangements exist in some PGs. 
Some irrigation groups do not pay the 7:1 share agreed upon (from 
the previous 9:1). 


Another illustration of such practices is mortgaging of land be- 
longing to members. When members take large loans (mostly for 
unproductive reasons) from the PG funds, their land is kept 
mortgaged, which the PG then uses for cultivation, and the member 
cannot get it back before a certain specified period of years. This is 
happening in all. the three regions visited. In essence, this practice 
implies that the member will not be able to get his land back from 
mortgage. 


Such unjust social practices need to be actively and strongly 
discouraged to ensure long-term viability of PGs as organizations of 
the poor. 


(c) Practices likely to weaken PGs 


Some practices in relation to PGs have come to our notice that are 
likely to weaken their organization unless checked immediately: 


(i) 


(ii) 


(iii) 


(iv) 


(vy) 


(vi 


More than two-thirds of the PGs visited did not have properly 
written down resolution books. This is particularly critical in 
matters relating to loans, rates of interests, sharing of benefits 
and profits, etc. 


Implications of decisions made by PGs that appear tobe unfair 
on the motivation of members is critical. The leadership of 
many groups seems to enjoy the fruits of being in a group more 
than the ordinary members. 


In most PGs visited, the chairman and secretary took loans, 
sometimes in greater amounts than the other members. 


Leadership of all PGs is in the hands of the most educated and 
competent members. However, these leaders are dominating 
the members, thereby reducing growth opportunities of others. 
The views of leaders normally prevail over other members. In 
many cases, these active, educated and competent persons 
continue to be in positions of formal leadership (chairman/sec- 
retary for periods of a:stretch of eight years in a row in some 
cases). 


The size of many: PGs in terms of membership has remained 
stagnant and small.-The PGs with 10 to 15 members are many 
in number, particularly landless (average membership 19) and 
women (average membership 18). Such small numbers are 
non-viable as far as bargaining power and struggle against 
social injustices is concerned. It is important that PGs obtain a 
collective strength of numbers if they want to influence the 
existing system. 


A related issue is that of coverage. In none of the villages (or 
hamlets) visited had the organizing effort covered all the poor 
families. In many cases, PGs had been formed in new villages 
even when many landless and marginal farmer families of pre- 
vious villages were left out. From the point-of-view of 
long-term viability and strength of the organization of PGs, 
large coverage of geographically contiguous areas is essential. 


While the record of accumulated savings has been impressive, 
it seems that savings orientation is the predominant activity of 
PGs. A lot of meeting time is spent on collecting savings. The 
knowledge of members is better regarding savings than any 
other activity of the PGs. Most members lacked an understand- 
ing of the empowering role of PGs, though they all knew its 
savings role. Even reporting by field workers and regions (ac- 
cording to a format prescribed by the central office) gives a 
predominant position to savings. 
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It is, therefore, desirable that less emphasis, relatively speak- 
ing, is given to savings and it is seen only as a means, not an 
end, of strengthening PGs. 


ISSUES 


Several challenging issues in the process of evaluation emerged from this 
experience. Some of these are highlighted here. 


1. One of the main issues related to the composition of the team of 
facilitators of the evaluation. The three-member team comprised of a 
woman social scientist of the region (to ensure a clear focus on organizing 
women and smooth rapport-building with poor women), an ex-staff 
member of CB (as one involved in previous evaluations and knowledge- 
able about CB, its staff and programmes, as well as trusted by them), and 


CRITERIA FOR ASSESSING PARTNER GROUPS 


These criteria were developed in workshops with the CB central office and field staff, 
and each PG was assessed on these by the concerned staff. | 


1. Composition 


(a) Minimum 15 members, maximum 30 (fixed membership). Ensure participation 
if larger number. 

(b) Regular annual general meeting and elections. 

(c) Capable board/committee (minimum three persons). 

(d) Homogeneous (commonality of interests) membership: sex, occupation. 


2. Income generating activities 


(a) Undertaking small pilot projects with their own funds. 

(b) Undertaking projects with the support of government programmes. 
(c) Undertaking projects with the support of other donor projects. 

(d) Noticeable individual and collective economic improvement. 

(e) Equitable distribution of resources among members. 

(f) Judicious use of own savings and local resources. 


3. Other actions 


(a) Concerted action against social injustices. 

(b) Cultural acttvities. 

(c) Collective action against superstitions and vices. 
(d) Being recognized locally. 


4. Linkage with other groups 
(a) Organizing campaigns and rallies. 
(b) Attending meetings of other groups. 
(c) Resolving local conflicts. 
(d) Having multiplier effect on other groups. 
(e) Interpersonal relationship characterized by mutual trust. 
(f) At least 10% literate members over a period of time. 


. ee ative of a resource agency (proposed by the donor). Building a 
ring relationship among the team of these facilitators was one of the 
main tasks in the early stages of evaluation. 


y The second issue related to the involvement of different levels of 
staff in CB. As a large and well established voluntary agency, the internal 
hierarchy of roles and responsibilities was well defined. To ensure free 
and open expression of opinions and perceptions by all staff, and to 
promote enhanced participation of field workers in the evaluation exer- 
cise, deliberate attempts were made to structure workshops of the same 
level of staff, and create distinct information sharing and analysis forums 
for different categories of staff, to minimize the effects of hierarchy. Such 
forums were created in the field as well as in the head office. 


Another method used in this regard was to ask various levels of staff to 
develop, individually and collectively, criteria for assessing the effective- 
ness and strength of partner groups (PGs). These criteria were then con- 


5. Organizational skills 
(a) Regular meetings. 
(b) Regular attendance. 
(c) Active participation of all members in discussions and decision-making. 
(d) Growth of effective and shared leadership and members’ control over it. 
(e) Deliberations recorded. 3 
(f) Follow-up of decisions made. 
(g) Regular savings. 
(h) Accounts clearly maintained. 
(i) Funds deposited in safe custody. 
(j) Procedures followed with rotating moderator. 
(k) Ability to withstand crisis. 
(I) Discussion of common and serious village issues in meetings. 


. Level of awareness 

(a) Identification of local money-lenders, touts, Union Parishad Chairman and — 
Members. : : 

(b) Existing local government facilities and programmes. 

(c) Land ownership and related systems and procedures. 
- (d) Local social system and prevailing social injustices (e.g., dowry) and their 

underlying causes. 

(e) Knowledge of legal rights (e.g., Minimum Wages Act, Dowry Act, etc.). 

(f) Importance of sending children to school (also sending them). 

(g) Knowledge of preventive health care. 

(h) Identification of class stratification. 

(i) Knowledge of what is going on in the country. 

(j) Knowledge of what is happening outside the country. 

(k) Encouragement to spouses to join groups. 

(l) Understanding implications of registration. 


. Training of members 
(a) At least one cadre from the group. 
(b) At least one member trained in some skills. 
(c) Having planned educational component in their activities. 
(d) Acquisition of skills to evaluate their own group. 
(e) Ability to plan and implement the project. 
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solidated and utilized for assessing PGs. The assessment of PGs on the 
basis of these criteria were first made by the field workers themselves, and 
then discussed with their supervisors, who had also made independent 
assessments of the PGs on the same criteria. This process also ensured 
widespread reflection and analysis among field workers. 


3. Another issue related to the process of building a collective group in CB 
as the main implementors of the evaluation exercise. The regional office 
and head office represented distinct units of CB. Since the programme 
was spread all over, it was felt important to include Regional Directors 
and their colleagues, alongwith key head office staff in the entire evalua- 
tion process. Thus a 25-person key-members’ group was established 
within CB to be involved in the evaluation exercise as key implementors. 
The setting up of objectives of evaluation was initially done by this group; 
the feedback of initial findings and tentative analysis was also shared with 
this group. Identifying and involving, these key members of CB individu- 
ally and collectively, was a critical aspect of the evaluation exercise. 


ANDHRA 
PRADESH 
BALWADI 
PROGRAMME 


BACKGROUND 


Andhra Pradesh Balwadi Programme (APBP) is a programme of non-for- 
mal education and supplementary nutrition being conducted by the Dio- 
cese in a backward district of Andhra Pradesh. The programme has been 
in existence for the past six years and has operated in about 200 villages of 
the district. The programme entails running of about 220 Balwadis for the 
pre-school children of the rural poor. A teacher runs the Balwadi, assisted 
by a Sevika. Children attending the Balwadi are given food for nutritional 
purposes. For every 20 Balwadis or so, a supervisor is available to guide 
the teachers. Balwadis have been used to help build rapport in the com- 
munity and improve the educational level of the children of the rural poor. 


An intervening agency was approached by the donor to evaluate this 
programme. The director of APBP subsequently held consultations with 
the intervening agency before the evaluation process started. 


PROCESS 


This evaluation exercise was carried out in three broad steps: 


1 An initial visit to the field by a member of the intervening agency 
was made for eight to 10 days. During this visit, the facilitator discussed 
the process of evaluation with the core team, parish priests and sisters, 
some supervisors and teachers of the Balwadi. 


The objectives of evaluation, the involvement of people in the evalua- 
tion process, the methods to be used were all decided along with the core 
team. It was decided that this core team will be responsible for carrying 
out bulk of the evaluation exercise, including initial data collection. 


A guideline was given by the facilitator to the core team to be used in 
the next three-four months when they were collecting information from 
parish priests and sisters, Balwadi teachers and supervisors, members of 
Catholic Sabha and Mahila Sangh. 
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After this, the core team met the parish priests as a group. Almost all the 
parish priests came for this meeting. A small questionnaire was also seni 
to the core team to be filled in by the parish priests. (All the parish priests 


replied, except one.) 


The questionnaire was also sent to the sisters but they did not reply 
because they felt that the Balwadi is a responsibility of the parish priests. 
They did not see their own role in the evaluation process. 


One of the core team members met the Balwadi teachers and super- 


visors, parish-wise and spoke to them about the evaluation process and 


their role in it. Some initial data collection was done from this group too. 


Efforts were also made by the core team members to meet the Mahila 
Sangh and the Catholic Sabha but this was possible only in four parishes. 


2 A team of two facilitators made the second visit after five months to 
visit Balwadis and meet the children and teachers. 


A day-long meeting of all Balwadi teachers and supervisors was or- 
ganized. Almost 120 teachers and supervisors (out of the total strength of 
220) came for the meeting. Twenty one parishes out of 28) were rep- 
resented. 

The teachers and supervisors were asked to individually write down 
five characteristics of a good Balwadi, discuss it in their own parish group 
and then present it to the larger group. A summarization led to a consoli- 
dated set of characteristics. The teachers then sat in their own groups, 
along with their supervisors, to see how far these characteristics were 
present in their own Balwadis. They did this exercise individually, then 
discussed it in their own groups. The groups also discussed how they 
could improve the situation in each of the Balwadis. After each group 
came to a consensus, it presented suggestions to the larger group. The 
facilitators tabulated these for the larger group, aggregated them and 
facilitated collective reflection on the data so generated. 


In one parish the facilitators were able to meet the Catholic Sabha 
members and Mahila Sangh members. Children of some of the members 
were attending the Balwadi. The discussions with these groups focussed 
upon the purposes and effectiveness of the Balwadi programme. 


The facilitators also interviewed the core team members in order to 
obtain their perception of the Balwadi programme. 


3A draft report of the evaluation process was then prepared by the 
facilitators and sent to the core team. This was followed by another visit 
by one of the facilitators to discuss the draft report. During this time the 
facilitator also met two additional groups of Mahila Sangh and a Catholic 
Sabha. The facilitator also visited three Balwadis and met some parish 
priests there. A final report was then prepared based on these delibera- 
tions and field visits. The core team has since used it to plan their future 
programme activities. 


KEY FINDINGS 


1. On the whole it was seen that children have benefited after attending 
the Balwadi 


Almost all the parish priests who replied to the questionnaire felt that the 
children had benefited from attending the Balwadi, though some felt that 
it served the purpose partially. Teachers, supervisors, Catholic Sabha and 
Mahila Sanghs also felt Balwadis had a positive effect on the children. On 
visiting the Balwadis one could see the enthusiasm and interest of the 
children. 


Effects on the children could be compared in the Balwadi itself. In the 
class, they would very eagerly talk and answer to questions; recite poems 
based on local themes, stories close to their life-situations; do simple 
addition and subtraction with the use of local aids; talk with confidence to 
facilitators who were outsiders. 


In comparison, the newer children were shy, afraid to talk, sat quietly 
in a corner, and the teacher had to make efforts to involve them in the 
class. 


2. Children going to Balwadis adjust easily into the formal school system 


Parish priests felt that children coming from Balwadis were mentally 
prepared to come to the school, hence could adjust easily. They learnt 
faster than others as they had gone through a systematized learning 
process. The teachers found it easier to teach these children and were 
appreciative of the Balwadi programme. 


3. Nutrition programme seemed to have benefited the health of the 
children to some extent 


Parents, Catholic Sabha, Mahila Sanghs and teachers feel that the quantity 
of food given under the nutrition programme should be more. They feel 
the present quantity is not enough and the child remains hungry. They 
also feel there should be more variety in food so that the children enjoy 
what they are eating, and do not get bored eating the same thing everyday. 


Some feelings were also expressed in terms of the type of food being 
given as to how similar it was to their normal eating habits. 


The staple food of the area is rice. In the Balwadis, children are given 
wheat preparations. Children do not like it and also the parents feel their 
children remain hungry. One of the villagers points out if the child gets 
used to some non-customary food.and new food habits, and develops a 
taste for that, how will the mother satisfy him/her later when he/she 
leaves the Balwadi. 


A local committee of three village men exists to look after the nutrition 
programme. Wheat or other such cereals are given by the parish. In the 
beginning it was planned that wheat would be bought by the villagers but 
later the parish priests felt that money could be saved if the wheat was 
bought in-bulk by them and then given to the villagers. The job of the 
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committee is to bring the wheat from the parish to the community. Other 
materials are to be bought from the market for which money is given to 
the committee. Since the members do not see it as their programme, they 
just implement whatever the parish priest says and do not take much 


interest. 


It is normally the mother’s role and duty to look after the child's health, 
but the local committee does not have any women. This also affects the 


functioning of this programme. 


The quantity of food given is not enough for older children and for 
those who come from far off villages whereas it is more for smaller chil- 
dren. There is a need to distribute the meal according to the needs of the 


children. 


4. Local people have not been involved in running the Balwadi 


It was seen that the Balwadi programme is largely viewed as an end in 
itself. Very little efforts have so far been made to integrate this programme 
with the other programmes of the parish. Each programme was seen as a 
separate unit. 


Local groups like the Catholic Sabha and Mahila Sangh exist but little 
efforts have been made to involve them in taking responsibility for the 
Balwadi programme. When members of the intervening agency met both 
these groups in three parishes they said they never had an opportunity to 
think as to what they could do and also no one had asked them. Since they 
had never thought of this before, they concretely wanted to know how 
they could help out and take responsibility. 


The core team has decided to make efforts to involve these groups. 


ISSUES 


Some of the interesting issues arising out of this experience were: 


1. The large size and widespread of the Balwadi programme posed a 
major question in planning the evaluation. Since the impact of the prog- 
ramme was the main focus of evaluation, it was felt necessary to cover, as 
widely as possible, the programme area and staff. Hence, much of the 
initial data collection responsibility had to be shared by the core team of 
the programme. Also it was necessary to design a simple questionnaire 
which could then be used to elicit information from all the Balwadi 
teachers. This meant that the intervening agency and its team of 
facilitators had to spend more time in the preparation of the core team mem- 
bers, both in terms of playing their role, and in developing the necessary 
skills needed for performing that role effectively. 


2. Another issue, related to the task of the intervening agency, was the 
special situation obtaining in a Diocese-sponsored development prog- 
ramme. The structure, norms and procedures, obtaining in the diocese are 
rather tightly laid down and followed, and the facilitators had no prior 
experience of the same. Even the hierarchy is so well defined, like between 


priests and sisters, that the facilitation of critical reflection was a difficult 
task. Separate events, informal conversations and structured question- 
naires —a wide variety of ways of collecting information —had to be utilized 


in order to develop a holistic understanding of the situation and to facili- 
tate a process of critical reflection among various categories of staff. 


37 


38 


RURAL 
DEVELOPMENT 


ORGANIZATION 


CONTEXT 


Rural Development Organization (RDO) began its work in the mid seven- 
ties with a focus on conscientization and organization of socially and 
economically deprived and exploited sections of the society. The major 
thrust of their initial work was to give a voice to the voiceless and power 
to the powerless. RDO started its work in 1975 through one field centre in 
five villages in one of the most backward districts of Madhya Pradesh. In 
1979-80, it established two other field centres and in 1984, it once again 
expanded into new areas. 


In fact, 1975-79, 1979-84 and 1984 onwards are three distinct phases in 
the development of RDO. During each of these phases, several internal 
and external evaluations of RDO took place, and a considerable amount of 
written material on its strategy, philosophy and programmes is available. 
It has come to be recognized in India and outside as a pioneer in 
operationalizing conscientization and organization model. With a suc-. 
cessful campaign of political education and building up of people’s or- 
ganization in the field areas during the first two phases (i.e. 1975-84), it has 
not only expanded its work in 350 villages, but has also started some 
socio-economic and health programmes alongwith people’s organization. 
The introduction of socio-economic programmes in the third phase started 
having some repercussions on the organizational functioning and work 
culture. 


At this juncture, at a meeting of RDO representatives and representa- 
tives of major funders of RDO, it was felt, if through a process of par- 
ticipatory evaluation a perspective and organizational strategy for the 
next ten years could be evolved, it would help in programme planning 
and facilitate composite funding for ten years. In this meeting, after a 
mutual agreement between RDO and its donors on the above need, an 
agreement was reached on the names of three potential agencies in India, 
one of which could play a facilitator role for the proposed process of 
participatory evaluation. 


By the time follow-up action was taken on the agreements reached at 
this meeting and the potential agencies contacted, RDO was facing some 
problems from the government and restrictions were put on the organiza- 
tion on receipt of funds; its records were sealed by government officials 
and an enquiry was set up. This intensified the brewing tension in the 


organization caused by the pressure of organizational functioning and 
conflicts of work culture in its third phase. Due to the’ uncertainty created 
by this new crisis, most of the people in the organization were apprehen- 
sive about the future, when the first planning meeting was organized 
between the intervening agency and the senior staff of RDO. The planning 
meeting was attended by a team from the intervening agency, representa- 
tives of other partners of RDO, a tentative representative of donors and 
senior staff of RDO. Apart from setting-up objectives of the proposed 
evaluation and developing the frame of reference, a detailed time 
schedule was also prepared. The objectives of evaluation derived in this 
meeting were as follows: — 


(i) To assess the model followed by RDO as a mechanism for people’s 
empowerment; 


(ii) To assess the relevance of different programmes in this model; and 


(iii) To assess the appropriateness of the organizational form of RDO. 


THE PROCESS 


With the above objectives in mind, it was felt that the evaluation should 
focus on the experiences of the model followed by RDO as a model of 
development (i.e. conscientization-organization model) and examine 
RDO’s programmes and organizational structures in that light. Thus, the 
essential thrust of the evaluation was not on assessing impact of the 
programmes on the beneficiaries, but on the viability and applicability of 
the model and its appropriate supportive structures. 


At the end of the meeting, some questions were raised regarding the 
suitability of the time-frame for the proposed evaluation process, in the 
light of the crisis RDO was facing. After careful thinking, based on the 
assessment of RDO staff present at the meeting, it was decided that the 
evaluation team would visit selected field areas and discuss the basic 
tenets of participatory evaluation with responsible area staff members 
and carry the evaluation process forward, taking stock of their reactions. 


As per this understanding, the evaluation teams visited the areas and 
held meetings with ‘the area staff. These meetings were quite useful in 
terms of building confidence among the staff members to candidly assess 
the need for a participatory evaluation and the appropriateness of the 
time-frame in conducting the same. In the course of the deliberations, it 
became clear that the staff was unmoved by the present crisis and was 
ready to join in the evaluation if their participation was ensured till the 
finalization of the report and planning of follow-up action. 


Some of the staff considered the proposed evaluation as an exercise to 
merely satisfy the formalities of the donors and their organizations. They 
said that this consideration was based on their past experiences when 
they neither saw the reports, nor saw any actions flowing out of earlier 
evaluations. However, with repeated assurances of discussing the draft 
report with them before finalizing it, the evaluation team won their confi- 
dence in some places; and in other places, some staff members were still 


39 


40° 


doubtful and wanted to assess the credibility of the evaluation team. This 
visit of the evaluation team was thus quite fruitful in creating an environ- 


ment for carrying out the evaluation process. 


After discussing these impressions with the Executive Director and the 
Acting Executive Director, it was decided to go ahead with the process. 
The dates for the next visit of different members of the evaluation team 
were finalized and the Acting Executive Director, who was made incharge 
of coordinating the work with the evaluation team, was informed. 


During the second visit, as scheduled, the scenario had drastically 
changed. No. one from RDO concerned with the evaluation process 
was available. One of the,evaluation team members proceeded to 
the area as per schedule on her own, and interviewed some of the staff. 
Another member reached afterwards but could not achieve much because 
of the prevailing state of confusion due to the legal crisis still facing RDO, 
coupled with flood relief work and some dynamics in the organization 
unknown to the evaluation team till then. Perhaps, the dynamics was 
escalating faster due to the prevailing uncertainty in the organization. 
After prolonged discussion with the Acting Director, the next visits were 
planned and an assurance was given that all arrangements would be 
made for the extensive field programme of the evaluation team. 


In the meantime the resource person involved in facilitating the evalua- 
tion process for the social forestry programme had fixed a meeting of 
concerned staff members in order to initiate the process. The internal 
conflicts and dynamics among senior RDO staff somehow stalled the 
process from moving further and the concerned ,resource person was 
forced to withdraw from the evaluation process. 


The interesting point at this juncture was that most of the responsible 
people in RDO were neither ready to take a stand to suspend the process 
of evaluation, nor were they comfortable about it continuing since they 
were not able to assess the possible outcomes of the exercise and its 
impact. With everybody overtly accepting the importance of participatory 
evaluation exercise, promising all cooperation in future, and apologeti- 
cally regretting the futility of the last two visits, it became a “’Catch-22” 
situation for the team from the intervening agency. The agency could 
neither back out from the exercise on its own initiative, nordo anything to 
expedite the process. ‘’Wait and watch” was the only line of action left to 
deal with the situation. 


In order to keep the records straight, the Acting Director of RDO was 
reminded of the coming visit of the team along with his assurances and 
two formats seeking some information. 


STRAINS ON THE PROCESS 


On the arrival of the evaluation team for the scheduled visit, the team, to 
its utter surprise was told that the process of evaluation was suspended at 
the instance of the donors as they wanted the process to continue only in 
the presence of their representatives. No information to this effect was 
sent to the intervening agency. On enquiry, it was revealed that the 


decision to suspend the evaluation was made a week before this visit of 


the team, but for reasons unknown till today, it was not communicated to 
the facilitating agency. 


This belated reaction of the representatives of donors on the issue of 
frames of reference and on the participation of donors further confounded 
the already prevailing air of mistrust and confusion within RDO regarding 
this evaluation. In fact, this intervention, leading to the postponement of 
the evaluation process, had severely affected the basic fabric of participa- 
tion based on mutual respect and trust among the concerned parties. The 
displeasure of the facilitating agency was duly communicated to all con- 
cerned parties. 


REVELATION OF AGENDAS 


After a lapse of two months a meeting was organized to sort out the 
misunderstanding among the concerned parties. But the meeting turned 
out to be a revelation of the covert purposes of evaluation which had not been 
made explicit till then. The issue of the appropriateness of the time being 
conducive for evaluation of RDO was not raised by anyone. Only the 
issue of the donors’ participation with the evaluation team during field 
visits for data-collection was raised. The arguments given in support of 
the donors’ participation in the whole process of evaluation implied that 
if evaluation is done in their absence, it would not be credible enough to 
convince the Board of Directors of the donor organizations and the philan- 
thropic organizations supporting them. This problem of convincing the 
Board of Directors of donors and philanthropic organizations was neither 
mentioned as one of the objectives of the proposed evaluation during the 
consultation nor was communicated prior to this meeting. 


The facilitating agency was taken by surprise on this matter and took a 
stand in this meeting that the involvement of the donor and philanthropic 
organizations in the evaluation exercise at this stage and the prevailing 
situation in RDO can be accepted till the headquarters level and at the 
stage of planning and finalizing the process only. The facilitating agency 
made it clear in this meeting that it had agreed to take up this task because 
the objectives of evaluation implied assisting RDO to take stock of its past 
experiences and to plan for the future and because of the mutual trust 
reposed in the agency by RDO and the donor organizations. Opening up 
of new frontiers at this late stage had belied both the premises of the 
involvement of the facilitating agency. At the same time, the facilitating 
agency was not interested in withdrawing its support to a fraternal organi- 
zation when it was under attack from the government. This dilemma was 
finally resolved through a compromise. It was decided that the donors 
would send their team separately, which would be joined by some rep- 
resentatives of the philanthropic organizations and some others. The 
findings would be shared with the facilitating agency. Somehow the 
deadlock was removed through this compromise. 
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NEW PHASE 


Sensing the problems caused by the six month long postponement, and 
the developments within the organization caused by external pressures 
and internal dynamics, the facilitating team decided to prepare question- 
naires for different levels of staff, village council members and formats for 
programme and village council profiles. The team had also prepared 
guidelines for indepth study of selected village councils. In effect, the 
team changed its method of collecting information from group discussion, 
collective and individual reflection (which were used in the first phase of 
evaluation) to relatively more structured methods of information collec- 
tion. 

But when the team reached RDO, it was confronted with a new situa- 
tion once again. In the briefing from the Executive Director, the team was 
told to extend the period of evaluation since the retrenchment of 
animators/teachers had created. new tensions in the organization. The 
team was told that teachers were given compensation according to rule 
and asked to leave and, as a result of dissatisfaction caused by such an 
action, they were spreading rumours among the people. The Executive 
Director felt that in such a situation, the information the team collected 
might be prejudiced. 

At this stage the team explained to him that during this visit they had 
planned to concentrate on collecting information from the staff only, and 
would not be holding group discussions with council members and other 
villagers. With this understanding, the team proceeded to the field. Dur- 
ing the team’s encounter with the staff, with whom the team had de- 
veloped a good rapport initially, they were asked a lot of questions regard- 
ing the validity of the present process of evaluation in the context of the 
happenings of the last few months. They were also doubtful about the 
locus standi of the team on the basis of their observations during the visit 
of the donors’ team. They put forward straight questions to the team 
about an assurance of implementation of recommendations made by this 
evaluation, even if they cooperated with it. This was quite a learning 
experience for the team in terms of understanding the:limitations of the 
evaluation. 


The deliberations with the staff members revealed that a lot of actions 
were being initiated by those managing RDO, regarding which the 
facilitating team had been kept in the dark. Such actions on the part of the 
management of RDO had changed the environment in the organization. 
There was polarization around the issue of termination of teachers and 
reorganization proposed by the management. Most of the staff were only 
interested in discussing these issues and Proposed to hold a general 
meeting for that purpose. 


The state of RDO at this stage was characterized by serious internal 
conflicts among senior staff and the Executive Director, extreme mutual 


mistrust, and strategizing and counter-strategizing among warring fac- 
tions. 


Hence during this visit as well, the team could onl carry out i 
/ its 
scheduled work for four days while the rest of the pve tne 2 the team 


simply observed the highly charged activity generated by the proposed 
meeting. Though this forced observation exercise was quite a learning 
experience for the team in terms of developing insights into the internal 
dynamics and decision-making process of the organization, and was an 
important input to the evaluation exercise, but as a consequence, the 
team’s renewed attempts for systematic data-collection became a casualty 
once again. 


At this stage, the evaluation team felt frustrated and doubtful about the 
intentions of the concerned parties regarding the completion of this exer- 
cise. The team raised the issue with concerned persons and requested 
them to raise the issue in the forthcoming RDO governing body meeting 
and let the facilitating agency know what the stand of the organization 
was at the earliest. The team proposed to hold a meeting of all senior and 
important staff of RDO to discuss the present state of affairs. The team felt 
that such a meeting, apart from being an exercise to close the evaluation 
process, might also help in overcoming the stalemate faced by the organi- 
zation then, and temporarily managing the conflicts. 


On receiving a green signal from them, such a meeting was finally held 
in which some critical decisions were taken to phase out the then crisis the 
organization was faced with. 


The above description gives an idea of stresses and strains which has 
an effect on realizing the expected levels of participation of staff in the 
evaluation process and the originally planned expected objectives of 
the evaluation. This account of events also brings out the limitations of an 
external facilitating agency in sustaining a participatory evaluation pro- 
cess. 


Based on the information collected in this process, a detailed report was 
prepared and presented at a meeting, which was attended by 40 persons 
including the RDO governing body members, senior and important staff 
of RDO and the evaluation team members. Summary observations of the 


report are given below. 


OVERALL FINDINGS 


1. According to the original vision of RDO, it was expected to distinguish 
between the organization committed to the promotion of people’s move- 
ment and the people’s organization created through its efforts. Such a 
conceptualization is visible in the initial documents, but it seems that this 
distinction got lost somewhere along the line. 


This appears to be the main issue affecting the growth of village coun- 
cils as stronger people’s organizations, effectiveness of programmes, and 
confusion regarding organizational structure and functioning at different 
levels. Over the years, the distinction between village councils as 
people’s movement and RDO as a promotion organization got blurred. 
As a result, people’s representatives and forums above cluster level got 
intermixed with RDO area staff; animators originally developed as leaders 
from village councils became identified, and treated, as RDO staff; the 
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strengthening of people’s organization, and appropriate structures ata 
broader level, beyond a certain area, could not take place. Many animators 
and other staff began to equate the RDO Central Office with the central 


decision-making forum of the people. 


It is this blurring of distinction that needs to be critically looked into and 
immediately clarified. Other actions can then follow from this. 


2. This has led to cooption of people’s leadership developed through 
investing a lot of energy, resources, and time into the formal structure of 
an organization performing a promotional role. Such induction of 
people’s leaders in an institutional structure has perhaps not only created 
dual aspirations among such staff but also helped nurture misunderstand- 
ing among village council members. The duality of aspirations had further 
affected the internal dynamics of the organization. 


On the one hand, village council animators and cluster coordinators 
were acting as spokespersons of the people’s organization; on the other 
hand, they were subjected to, and became socialized into, RDO organiza- 
tional structure and style of functioning within the overall context. 


Many of these people experienced tensions and conflicts within them- 
selves around this duality; and acting on that confusion led to interming- 
ling of roles—acting as people’s representatives in staff meetings and as 
RDO staff in village council and cluster meetings. 


3. Rapid programmatic and.geographic expansion of RDO activities 
in its Third Phase (1984) further accentuated this situation. The demands 
for such expansion implied transfer of senior and experienced staff to start 
work in new areas, recruitment of technical and professional staff, and a 
large number of new field staff. This expansion appears not to have been 
planned very well and as a result, necessary orientation and integration of 
new staff could not take place. The third phase of RDO activities could not 
be comprehended by the new staff within the historical context of the 
earlier two phases; and the old staff, equipped to function effectively in those 
previous phases, could not be reoriented to take on the responsibilities of 
the new phase. 


The growth of RDO, apparently less than well planned, further shar- 
pened internal and external contradictions and conflicts. 


4. The organizational and personnel management style evolved dur- 
ing the process of working in a relatively smaller area, with focus on 
conscientization, organization and action was not reviewed and changed 
to adjust to the requirements of a large organization taking up develop- 
ment programmes also. Even the older staff members were not prepared 
or trained in planning, proposal developing, monitoring etc. 


The structures of decision-making, participation, mutual interaction 
and consultation that were appropriate during the previous phases were. 
no longer fully relevant in the changed context. Similarly, planning 
budgetting and control systems, previously appropriate, were not revised 
in the light of the new demands of tasks and people. 


5. The strategy adapted to cope with the new programmes in the 
expansion phase was to introduce the Technical Team model. Such a 
change in structure in a social organization needs to be carefully discussed 
so that the people involved could internalize it and contribute to enhanc- 
ing the effectiveness of the organization. Somehow it seems there is a lack 
of clarity about the objectives of such an important change in structure. 


As is explained in some of the documents, the reason for such a change 
was dual in nature, i.e. to deal with the growing regional feelings, and to 
cope with the skill development requirements of the new thrust. The net 
result was resistance from the people and older staff in areas. This change 
could be fully effected in only one among the three old areas, and another 
area developed a via-media of maintaining people’s structure alongwith 
Technical Team model. 


The changes brought about in this structure appear to have contn- 
buted to further blurring of the boundary between People’s Organization 
and Promotional Organization. The new demands perhaps necessitated 
changes in RDO staff structure, but the net effect was a change in people's 
organizational structure. 


6. The major impact of such a change was on horizontal communica- 
tion between people’s bodies, which seems essential for consolidation of 
people’s movement. With the introduction of the Technical Team model, 
no natural forum was left for the village council members of one village to 
meet their counterparts from another village in the same cluster/area, at 
their own initiative. The only communication channel left open was 
through RDO staff. This placed the responsibility of running and con- 
solidating people’s organization on RDO without much participation of 
people’s leaders. 


Another effect of this change was felt on those who were village council 
members in the past and were first made animators and then regional 
coordinators, or on those who were village council leaders first and then 
coopted as regional coordinators. These people, when asked to acquire 
some or the other skill in order to change their role to Technical Team 
members, found it difficult to cope with it in the short span of time available. 
As aresult, it appears to have overloaded them in terms of coverage; 
also it did not match their personal goals based on the dual aspirations 
referred to earlier. The resistance caused by this process of change has 
affected the functioning of the organization. This has also led to mistrust 
and suspicion among the staff at different levels, and polarization of 
different categories of people in RDO and village councils. 


7. The organizational and personnel management style referred to 
earlier, in the highly polarized and changed context of these develop- 
ments, led to a decline of democratic norms of organizational functioning. 
The role of a few committees left with people’s representatives, i.e. steer- 
ing committees, in decision-making gradually declined. The deci- 
sion-making powers, for all practical purposes, were concentrated at the 
top. Detailed information regarding planned proposals and budgets ini- 
tially used to be taken to the village council and discussed in order tovset 
up democratic norms in the organization. Now, they had stopped even 
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reaching the staff. This style of management developed in all areas, in- 
cluding personnel, financial, administrative and inter-personal aspects. 


Most of the area coordinators have also tried to imitate the same style in 
their areas, which has made things worse. The only mode of information 
flow developed due to such a style reflected a ‘closed door policy’ ; 
wherein only the policy makers knew of the various related issues, while 
all others were in the dark. As things stand now, the flow of information 
has become very restricted and personalized. This has also contributed to 
speculation, rumour, distortion, and inter-personal mistrust and distanc- 
ing. The prevailing system of information flow and decision-making could 
not cope with the changing demands of the new phase, and over-centrali- 
zation became the response to the resulting chaos and confusion. 


The development of various economic and social programmes did not 
occur on the explicit premise of continuously strengthening people’s 
organization and movement. As a result, many economic and skill build- 
ing programmes have been initiated in such a manner and at such a pace 
that the promotional role of RDO (in conscientization and mobilization) 
got gradually replaced by service-provider role. 


The loan programme, in particular, became the focal point of village 
councils, and its expansion, as opposed to that of village council member- 
ship and strength, took precedence. 


Evolution and use of various socio-economic programmes in support of 
strengthening people’s organization and movement somehow-got dif- 
fused. And, as a result, programmes became the primary task, not the 
village councils and their higher level structures. 


The programme was conceived and implemented in all the areas — old 
and new — almost uniformly, without reference to specific requirements 
and needs of a particular village council, region or area. The stereotyping 
of programming had further reduced the possibility of utilizing the prog-. 
rammes to strengthen people’s organization. 


In sum, the RDO experience demonstrates the viability, and even de- 
sirability, of building people’s organizations, and village councils as their 
base structures. The programmes and structures of RDO promotional 
organization need to be distinguished from the people’s organization and 
movement. It is in rearticulation; and consequent reformation, of this 
strategy that the future direction of people’s.organization and RDO as a 
promotional organization should be attempted. 


These observations were discussed at length during the meeting and 
there was a general agreement on all the points. It was resolved in that 
meeting to translate the whole report in Hindi and discuss it at various 
levels in the organization. Planning of the action to be taken was differed 
in this light, to be done after the discussion. 


The intervening agency did not receive any communication after this, 
hence its role came to an end at this stage. 


ISSUES 


This experience brings out a number of issues related to the understand- 
ing of participatory evaluation. These issues on one hand highlight some 
essential points to be followed in a participatory evaluation exercise and 
on the other, illuminate some limitations of participatory evaluation. We 


are attempting to put down some of these issues here for the purpose of 
further discussion. 


1. Explicit statement of objectives of a proposed participatory evaluation 
exercise by all the concerned parties (constituencies) related to the organi- 
zation’s wark is essential for carrying out the participatory process. Any 
room for possible reinterpretation of objectives in between or imposition 
of new objectives by one or a few parties alone, is bound . to hamper the 
participatory process. Hence, it is better to initially spend some more time 
in setting up the objectives, explanation and interpretation of each objec- 
tive etc. with all the concerned parties, in order to ensure an uninterrupted 
participatory process later on. 


2. Working out a detailed “memorandum of understanding” acceptable 
to all concerned is also essential, however bureaucratic it'may sound on 
the face of it. Such an understanding may entail roles and responsibilities 
of each party and individuals involved in the participatory evaluation exer- 
cise. It may also incorporate an understanding regarding communication 
of all important decisions taken in the organization during the time of the 
participatory evaluation exercise to all concerned. On the whole, based on 
the merits of the case in hand, it may incorporate all those details regard- 
ing the understanding among the concerned constituencies which have 
potentials of distorting or interrupting the participatory process. 


This practice is quite common in sectors other than ours. Due to our 
utter dislike for structured and formal functioning we tend to generally 
ignore working meticulously on this aspect. Since, most organizations in 
our sector also operate in a multiple constituency systems and the par- 
ticlpatory evaluation process entails involvement of most of the con- 
stituencies the organization deals with, it is always better to create a clarity 
of understanding at the outset. The elements of misunderstanding, mis- 
trust and confusion among the- cdncerned constituencies cropping up 
during the process defeats the whole purpose; hence it is essential for us 
to come to terms with this bit of formalism. Our experiences in recent 
years indicate a welcome trend of organizations initiating participatory 
evaluation on their own, without anybody asking them. This trend needs 
to be encouraged further. 


3. Another issue relates to the stakes which the organization has in 
evaluation and its impact on participatory evaluation. Participatory evalu- 
ation is an exercise of critical reflection leading to strengthening the vision, 
organization and programmes. A situation where there are no stakes 
involved will doubtlessly be the ideal situation for such critical reflections. 
But in practice such ideal situations often do not exist. Hence participatory 
evaluation faces the challenge of coming to terms with, or coping with, 
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the actual situation of vested interests in any proposed exercise The 
experience shows that the two points mentioned above, regarding explicit 
statement of objectives, and formulating memorandum of understanding, 
do help in coping with such a challenge. It was found that such efforts 
may yield the desired result at the top level of the organization which on 
its own do not percolate to various levels. Hence some cautious, deliberate 
and effective efforts need to be built in the process itself, so that the 
restricting effect of the stakes in evaluation on the process of critical 
reflection could be minimized at all levels of the organization and among 


all constituencies involved. 


4. The environment in the organization is another issue of concern in the 
context of participatory evaluation. The case presented here had an or- 
ganizational environment of tension and mistrust which got polarized 
during the process of evaluation due to external factors and leadership 


response to those factors. 


Experience in this case and some other cases shows that in such a 
situation the process of participatory evaluation can be very helpful in 
coping with the situation if the organization’s functioning is non-bureau- 
cratic and the structure is non-hierarchical. But in a hierarchical and 
bureaucratic organization, even in normal circumstances it becomes dif- 
ficult to carry out a participatory process. Though it is hard to say at this 
stage whether this is a strength or weakness of participatory methodol- 
ogy, but certainly coping with the inherited resistance of bureaucratic and 
hierarchial system is a challenge which we face in varying degrees at 
various times. 


5. If we agree that any participatory evaluation exercise should essen- 
tially result in action for change, the question of the organization’s ‘wil- 
lingness to act’ becomes very important. In the absence of such a will, 
even a participatory exercise will tend to become routine; particularly if it 
deals with organizational issues of perspective, structures and work cul- 
ture etc. The action for change in this context amounts to organizational 
development intervention and thus, even the role of the intervening 
agency becomes crucial. So the willingness and readiness of the organiza- 
tion and the acceptance and competence of the intervening agency holds 
the key for such desired organizational development intervention. 


6. In a participatory evaluation exercise, the choice of appropriate 
techniques of data/information collection is another crucial area. Perhaps 
the rationale for the choice of techniques should be to promote the in- 
volvement of the people at various levels of the organization’s work. This 
makes it imperative that the choice of technique for different levels has to 
be different. The selected technique has to be appropriate to the level it is 
going to be used at. Hence, flexibility in the choice of techniques is essential 
for a successful participatory process. 


This requirement places extra demands on the facilitating team:of hav- 
ing a command over all the techniques, so that it could use appropriate 
ones whenever and wherever required. 


JAGRITI 


CONTEXT 


Jagriti is a voluntary agency with a team of seven people working with the 
tribals in Maharashtra for the past three and a half years. The organization 
was set up by four persons who came to this area with a view to work 
among the local tribals. The area is very backward, poor and comprises 
hilly terrain. After initial exploration and building rapport with the local 
people, the core team of Jagriti began working on a series of small activities 
including adult education centres, non-formal education and taking up of 
issues of local people with government officials. Several important strug- 
gles have been launched in the course of this period and in many of the 14 
villages where Jagriti works, strong local people’s committees have 
emerged which are beginning to analyse their situation and take initia- 
tives for their own improvement. While the work of the initial team has 
now been going on for five years, it is only in the past two and a half years 
that regular funding for its programmes became possible. 


PROCESS 


The donor agency supporting Jagriti asked it to seek the help of some 
outside agency to evaluate its programme, in particular its future direc- 
tions, so that decisions about the scope of future funding could be made. 
The two then approached the intervening agency jointly. It was discus- 
sed that the role of the intervening agency should be to assist in the 
process of reflection on Jagriti, in particular its internal organizational 
mechanism. It was suggested by the facilitators that this evaluation exer- 
cise should not be seen as linked with the future funding of Jagriti. It was 
then clarified by the donor that it was committed to supporting Jagriti in 
the future, so the question of future funding is not whether funding will 
take place or not, but for what. The intervening agency then agreed to 
facilitate this process. 


An initial planning workshop was held with two other voluntary agen- 
cies from the area interested in undertaking participatory evaluation for 
themselves. In this workshop, general principles and philosophy of par- 
ticipatory evaluation were discussed and each organization evolved its 
own plans for evaluation. The facilitators’ team helped in setting of evalua- 
tion objectives and methods of data-collection and analysis necessary to 
accomplish the same. 


Subsequent to this workshop the facilitators’ team made a field visit to 
the area of work of Jagriti and observed its programmes in the field, talked 
to some members of the local communities, but spent most of the time 
discussing the work of Jagriti, individually and collectively, with the seven 
full-time staff of Jagriti. More time was spent with senior and original core 
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team members in helping them reflect on the experiences of the past ee 
and a half years and identifying issues of concern that need to be resolve 

for the future. From this, a list of issues was prepared which then became 
the basis for a second round of reflection in the field, after a few months. 


During the second round of field visit, most of the time was spent in 
reflecting with the core team and the rest of the staff on the issues which 
were identified previously. The bulk of these issues related to the organi- 
zational structure and functioning of Jagriti and its future directions. 
Based on these deliberations, discussions and analysis, some suggestions 
for the future direction were agreed upon. These are reported as follow: 


SUGGESTIONS FOR FUTURE 


(i) Need for redefining the objectives 
The present set of objectives of Jagriti are more a statement of ac- 
tivities than the objectives that one could accomplish. As a consequ- 
ence, Jagriti gets bogged down in completion of those activities and 
not accomplishing a broad set of objectives. 


It appears that the objectives of Jagriti could be broadly laid down 
so that it could accomplish them through a variety of activities, in- 
cluding the ones currently undertaken. It is suggested that Jagriti 
should have its objectives as “utilizing educational and organiza- 
tional strategies for socio-economic improvements of the tribals’ 
This will indicate socio-economic improvements of tribals as long 
term goals to be accomplished primarily through educational and 
organization-building efforts. 


(ii) Need for integrating new programmes 


Jagriti has recently begun several new activities with the help of aid. 
These include land improvement programme, construction of a train- 
ing centre and building a farm for self-sufficiency. 


Land alienation appears to bé a general problem of tribals in these 
areas. Jagriti should take this up seriously if it wants tribals to improve 
their economic situation. Hence,,it may become a major activity in 
different villages. 


However, the training centre and the farm need to be clearly plan- 
ned and integrated with the field work of Jagriti. It is important to 
ensure that building of the training centre and development of the 
farm does not imply that trained and experienced field staff withdraw 
from field work; and creation of this institutional infrastructure does 
not become a liability to the ongoing educational and organizational 
work of the tribals in the villages. 


(iii) Need for redesigning the structure 


As mentioned earlier, the main weakness of Jagriti relates to its exist- 
ing organizational structure. For one, the structure in its present 
shape appears to have served a useful function in the early stages of 
the work of Jagriti. At this present stage, when, consolidation is 


important, this structure requires several modifications as indicated 
below; 


(a) The existing governing body of Jagriti is internal and comprises 
some staff and local villagers. It is perhaps important to distin- 
guish Jagriti from the local people’s organizations. The two can 
have mutually supportive and complementary roles, but are not 
identical. It appears that the motivations, supervision and de- 
velopment of staff of Jagriti has been hampered by the structure 
of its present governing body. 


It is suggested that the governing body of Jagriti should com- 
prise external members who are not full-time staff. These persons 
can bring in objectivity, guidance, linkages with outside re- 
sources and provide a more broad-based character to the organi- 
zation. A small governing body, of say five persons, would be 
desirable. The secretary of Jagriti can be a full-time staff and 
ex-officio member of the governing body, acting as its secretary. 


(b 


~~ 


Existing field staff should each be assigned a particular area with 
a set of villages and asked to take complete responsibility for the 
same. Thus the staff should have area-wise focus, and relevant 
activities for that area can be evolved. For the time being, the 
secretary can play the role of field coordinator both by visiting the 
field (each area at least once a month) and by chairing monthly 
review and planning meetings. 


(c 


— 


The secretary is expected to perform the following support and 
problem-solving roles: 


— attend monthly area level meetings; 

— guide in solving local problems; 

— deal with the government; 

— liaison with the outside world (including fupders); 
— administrative supervision; 

— secretary to the governing body. 


(d) The staff of Jagriti has considerable experience now but lack 
systematic training. There is a need for them to develop the 
context, content and methods of this type of work, with a 
long-term perspective. It is necessary that general training for 
field staff be urgently organized. 


Besides, specialized training is necessary as well. The staff 
which is going to be responsible for the training centre needs to 
develop skills as trainers. The secretary needs to be equipped 
with a number of additional skills for ensuring ongoing, smooth 

' functioning of the organization. 


Field visits to other groups and organizations can be useful as 
a learning experience. Resources should be separately earmarked 
for such a training. 


(iv) Finances 
The core funding for Jagriti is presently coming from one donor. In 


the light of the additional staff required and expanding scope of 
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activities, Jagriti should explore additional funding as well. Itis neces- 
sary to secure ongoing bases of funding for the core staff and activities 
of Jagriti over the next three to five years’ period so that the begin-- 
nings made so far can be further consolidated and expanded. 


The proposed organization structure suggests that 11 full-time staff 
are needed in Jagriti. This may also need to be considered by the new 
governing body. The current salary structure of the staff has some 
anomalies, with a couple of people being relatively underpaid. It is 
suggested that the new governing body should take this matter ur--. 
gently and rationalize the salary structure. 


PROPOSED ORGANIZATIONAL 
STRUCTURE OF JAGRITI 


GOVERNING BODY (External) 


Secretary 
Office Assistant 
— Farm Manager — Coordinator — Incharge 
— Assistant Women’s Training Centre 
Programme — Assistant 
Field Staff Field Staff Field Staff Field Staff 


e: Existing staff is seven: four field, one Programme Coordinator, one Assistant 


and one Secretary. 


Not 


Total Staff needed: 11 
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ISSUES 


The main issues emerging from the Jagriti evaluation are as follows: 


1. The focus of the evaluation of Jagriti was initially not very clear. In the 
planning workshop, senior Jagriti staff talked about the need for 
assessing the impact of its work over the past 2-3 years. The short 
duration of their involvement in the area, with a small team and 
limited programme, made it difficult to assess whether the prog- 
ramme had made any difference or not. It was only in the first field 
visit that the focus of evaluation emerged more clearly. When the 
members of Jagriti started listing the issues of concern to them, then 
it became evident that the focus was on strategy and organizational 
mechanism of Jagriti. 


2. In this case, the experience of negotiating with the donor became an 
interesting issue. When the donor initially approached the interven- 
ing agency, it was looking at the proposed evaluation as a basis to 
decide whether to continue funding the programme or not. The dis- 
cussions with the donor and Jagriti together made it clear that the 
issue of funding may seriously hinder the critical reflection process by 
Jagriti. It was then emphasized by the intervening agency that the 
donor and Jagriti must sort out the issue of future funding, and delink 
it (especially whether or not to continue funding) from the evaluation 
process. Only then can a serious reflection process be initiated and 
sustained. 


INTER SCHOOL 
PROJECT 


CONTEXT 


In late 1976, as part of a scheme of the Board of Secondary Education, 
Bombay, ninth and tenth class students from private schools were in- 
volved in a non-formal education (NFE) programme where these students 
— teachers of non-school-going children (age 5-16) in the urban 
slums. 


At the time of the evaluation, 15 centres for slum children in the Inter 
School Project (about five each by three participating schools) were run- 
ning. 


Each centre consisted of about 30 learners in the Inter School Project 
(ISP). The centres were run six days a week for 1'/, hours each day. The 
student teachers worked in small groups (about 40 from each school) 
and were supervised by an adult Non-Formal Education worker at- 
tached to each school. Project staff also comprised two professional 
coordinators, a manager and a typist-secretary. The project director 
approached two facilitators to discuss the possibility of undertaking 
the evaluation, over the next seven months. The different steps in the 
evaluation process are shown below. 


STEPS IN THE EVALUATION PROCESS 


Se. : 1. The project director gave an overview of the 

Initial discussions | project to the consultants. The process of par- 

with project ticipatory evaluation was broadly discussed. The 
director. next two steps were planned. 


2. The facilitators visited five centres with the 
Planning for project director and supervisors just to 
evaluation with familiarize themselves with the activities. The 
project staff. sample centres represented the range of popula- 

| tion of the entire set of centres. Day long, visit. 


3. This day long event was utilized to jointly plan 
the evaluation. The two project coordinators and 
all the supervisors were present besides the man- 
ager. It is here that the objectives of evaluation 
were set, the multiple constituencies were iden- 
tified and the type of information/data required 
‘f from these constituencies were identified. Next 
two steps planned. 


Planning for 
evalpation with 
project staff. 

/ { 4 
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4. Half day exercise to discuss the objectives and 
tentative plans of evaluation with volunteer 
teachers leading to elaboration and modification. 
Two-day event planned with all volunteer 
teachers to initiate the evaluation process. 


Discussions with 
volunteer teachers 
regarding evaluation 
plan. 


5. The evaluation plan, and in particular 
data-collection plan, refined and finalized with 
Refinement of data | project staff. The instruments needed for data 
collection plan and | collection and alternative methods identified and 
identification of | responsibility for data collection and instrument 
methods with =| design allocated. The design of instrument as 
project staff. well as data collection was shared between pro- 
ject coordinators, supervisors, volunteer 
teachers, student teachers and consultants. 


6. The instruments were finalized in the joint 
meeting of all project staff and plans for data-col- 
lection in the Inter School Project made. 


Finalization of 
instruments for 
Inter School Project. 


7. As per plan, data-collection began and was 
completed in a month as the schools were closing 
for summer vacation. 


Data collection for 
Inter School Project. 


8. A facilitator along with the non-formal educa- 
tion worker of the school held discussions with a 
few student teachers to involve them in the 
evaluation. The objectives and plans for evalua- 
tion were explained and they were invited to join 
in the evaluation process. The event was also 
used to collect data through group interviews. 


Discussions with 
student teachers, 
analysis and feed- 
back evolved. 


9. Along with project staff, a plan for data 
Plan for data analysis and feedback evolved. All parties were 

analysis assigned responsibility for preliminary analysis 
of data collected by them. 


10. Data was analysed individually first and then 
a collective (with project staff) analysis session 
was held. Feedback meetings within schools 
were planned. 


Individual and 
collective data 
analysis. 
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11. Principal, representatives of principals, 
teachers, NFE workers and project staff were pre- 
sent in a half-day feedback meeting where salient 
findings were presented and critical issues iden- 
tified and discussed. A second meeting was also 
held. 


Feedback 


Report writing 


incorporating | 12. The suggestions made during feedback 
feedback discussions.] Meeting were incorporated in the report. 


Discussions on 13. Held: 
findings and process 
with project staff. 


SALIENT FINDINGS 


1. The study showed that, on the whole , the student teachers had bene- 
fited from their involvement in the NFE project. 


A questionnaire was administered to the student-teachers in the three 
schools. The data showed that a majority of the students in the three 
schools enjoyed working on the project and expressed interest in teaching 
children. Thus, 94 per cent of the student teachers in School A (N = 47), 
80 per cent in School B (N = 31) and 95 per cent in School C (N = 21) 
expressed this opinion. 


Most student teachers also felt that they had benefited personally from 
their participation in the programme. Some of the changes noticed by 
them were: 


“Tam now concerned about the poor people. I want to help them in 
any small way I can.” 


The teaching in the centre has changed me completely. I have 
become aware of the problems that teachers face while teaching. It 
has broadened my opinion on need for education.” 


“Yes, I have changed a little. For instance, I do not look upon the 
slum children indifferently and now try to understand their prob- 
lems.” 


“T have come to know a fraction of those living below the poverty line 
better than what I knew about them earlier.” 
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“We have come to know about poverty in India. 


“Mt has made me feel the intensity of the problems India is presently 
facing. The direct exposure has made me aware of certain problems 
personal, economic, etc., which I had never thought of earlier. 


A majority of the parents had also noticed a favourable change in their 
children: 46.7 per cent in School A (N = 43) and 65 per cent in School B (N 
= 20). Some of the changes observed were: 


“There are marked changes in her temperament and approach to 
human problems.” 


“She is more patient, more cooperative, more helpful, more under- 
standing than before.” 


“No apparent changes noticed, but I feel the changes will come and 
the experience and training will make her grow into a socially aware 
person.” 


In addition, the class teachérs and the NFE workers had also noticed 
positive changes in the student teachers. 


2. On the whole, the study indicated that the school teachers were not 
well informed about the NFE project. 


In each school, the teachers’ incharge, the class teachers and a sample of 
other teachers were administered a questionnaire. Their responses indi- 
cated that the teachers were not very clear about the objectives of the NFE 
project nor about the communities in which the students were working 
nor of the other schools that were involved in the project. There was also 
a lack of clear understanding about the aims and objectives of the project 
itself as part of the school curriculum. This was true even for the teachers 
incharge. 


In terms of their attitude towards the project too, a large number of 
them fended to be indifferent towards the project. Even a large number of 
students perceived their indifference to the project — 74 per cent in School 
C(N = 21), and 50 per cent in School B (N = 47). Among the suggestions 
offered by the teachers for making the NFE project more effective in the 
school were. 


“Teachers in general should be made more aware of the project, 
especially the class teachers.” 


“An awareness must be brought about and the whole school as a 
body must know about it and work towards it through different 
means.” 


3. The three principals showed interest in and commitment to theidea of 
NFE work. 


Indepth interviews with the principals/vice-principals indicated that there 
was full commitment to the aims and objectives of the NFE project. A 


need had been felt by the top school managers to start community sei vice 
for students. The schools became involved in the project as it was felt that 
it would have a positive impact on the minds of the children or else they 
would never understand the problems of poverty in the country. The 


project's achievements so far were summarized by one principal thus: 


“What the students could not get from standards one to 10, they can 
get in one year”, 


indicating thereby the positive impact of the programme on the school 
children. 


4. The study indicated that the NFE project had some positive impact on 
learners. 


One of the main findings here related to the fact that almost all the 
learners interviewed enjoyed being at the NFE Centre. They differed in 
their preferences for specific activities but they liked the wide variety of 
teaching methods such as drama, singing, games, outings, study, etc., 
that were used. 


Overwhelmingly favourable reaction was expressed by them about 
student teachers who, they said, “taught lovingly’’. All the NFE workers 
were also praised by the learners. 


The parents of learners felt the same way. They had only favourable 
opinions about student teachers and NFE workers. 


5. The role of the NFE worker is such that she faces multiple and conflict- 
ing demands. 


The NFE worker in the project has a complex and difficult role (see Fig. 3). 

She is simultaneously dealing with such diverse constituencies as local 

communities, parents of learners, student teachers, other teachers, princi- 

pals and other project staff. This present multiple and, at times, conflicting 

demands about her work. Thus an NFE worker needs to possess a range 
of skills to deal with such diverse constituencies. 


Another aspect of this role is the apparent confusion about the identity of 
the NFE worker. Is she a part of the school system or the project organiza- 
tion or both? Is she inside the school or outside it? If she is outside the 
school, how does she influence the school system? If she is inside the 
school, does she perform other tasks of the school? In particular, School A 
has resolved this problem by taking on the NFE worker as its full time 
staff. Thus she performs multiple responsibilities, besides NFE project, as 
part of the school faculty. In other two schools, this tension is still continu- 


ing. 


6. There had not been much systematic interaction between schools in 
the implementation of the project. 


Interviews with student teachers, NFE members and project staff, school 
social workers, and principals indicated that while there had been some 
‘nteraction between the NFE workers and among students of the three 
schools during the training programmes, the three schools had tended, by 
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and large, to work independently of one another. Suggestions included 
meetings between the three schools and sharing of experiences so that the 
schools could learn from one another and further strengthen the function- 


ing of the project. 


7. On the whole, student teachers had benefited more than children in 
the community and the community at large. 


Interviews with principals, school social workers, NFE workers, project 
staff and some community members seemed to indicate that despite the 
fact that positive changes had been perceived in the learners, yet the NFE 
project had.tended to give priority to the needs of the student teachers 
rather than the needs of the learners or the needs of the community. 


ISSUES 


Several issues emerged from this experience: 


1. Akey issue in this evaluation process was that of timing. The process 
of evaluation started in February, and the entire data collection exercise 
had to be over by April, before the schools closed for summer holidays. 
This put a considerable strain on the process and stretched the capacities 
of the project staff. The facilitating agency also had to make substantial 
time commitments in one lot. 


The summer holidays were used for analysis and discussion within the 
project staff. The feedback to the schools, their principals, teachers and 
others had to be organized soon after the schools reopened. This helped 
in implementing some of the decisions which resulted from the feedback 
meetings. 


2. The second issue in this evaluation was that of managing multiple 
constituencies. The NFE worker's role got translated into the evaluation 
process, as the project itself spanned several constituencies, besides the 
slum communities, students, teachers, principals/administrators, parents 
of students —a geographically, culturally and conceptually varied set of 
constituencies. Each constituency has its own purposes, its own percep- 
tions of evaluation, its own pace and space, its own preferences. The 
process of evaluation had to understand, and relate to, each of these 
constituencies and involve them in the evaluation. The task of managing 
the involvement of this diverse set of canstituencies in the evaluation 
process proved to be quite a difficult-one. 
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TRIBAL 
DEVELOPMENT 
SOCIETY 


BACKGROUND 


Tribal Development Society (TDS) is a field based voluntary organization 
working among the tribals in the backward regions of southern Bihar for 
the last eight years. It started its activities with a primary health care 
programme which essentially focussed on providing curative extension 
services for tuberculosis (TB), malaria and other diseases prevalent in the 
tribal villages. This work provided a legitimate entry for TDS into the 
tribal communities. Through this work, TDS gradually learnt about the 
problems and the forces that marginalized the tribals in the region. The 
local moneylenders and liquor merchants had captured all the assets 
(land, trees, cattle, utensils, implements, etc.) that belonged to the tribals. 
Over several years, the tribals became indebted and in turn bonded to the 
local moneylenders and liquor merchants. 


The initial core team of six persons working with TDS was disturbed 
by the indebtedness, bondage, and land and other asset alienation. The 
core team discussed the issue with the local tribals and started mobilizing 
people with the prime objective of empowering the community so that 
they can organize themselves to combat the exploitative forces. Over a 
period of two years the mobilization efforts developed into a large scale 
people’s movement. In this process, about ten thousand tribal families 
from 90 tribal villages started coming together. This effort led to the 
formation. of the Tribal People’s Organization (TPO) in the area, with its 
own local leadership. 


Along with the process of building Tribal People’s Organization and 
building community leadership, TDS assisted the tribals by taking advan- 
tage of the existing government programmes and legislations, in their 
struggle to get back their alienated assets. TDS’s effort has helped the 
community to redeem itself from economic exploitation. Once the assets 
of the tribals came back to them, the need for credit both for consumption 
and for production became apparent. TDS helped the tribal families to 
acquire credit from local banks and thereby revitalize their economy. 


During the first five years of work among the tribals, TDS was involved 
in a lot of programmes, but ‘in terms.of direction of work and organiza- 
tional operation, TDS was not clear what path it should take. There was a 
lack of clarity on whether they should continue helping the tribals or not. 


If they continue, then how? Should they undertake some developmental 
programmes with the assistance of the government? At this point, the 
organization was functioning essentially as a small core group of 10 per- 
sons (which included the original founders) and a local team of 10 
animators. The organization as a whole was functioning like a family with 
an informal and personalized style of decision-making. The lack of clarity 
in terms of direction of work, added to personal and informal deci- 
sion-making practices, created a sense of dissatisfaction, stagnation and 
frustration among the core group as well as among the new support 
colleagues. At this point TDS felt the need to undergo a process of critical 
reflection about its past in order to derive directions for its work and 
strength for its organizational functioning, and for its future. 


PROCESS 


A team of two facilitators was consulted by the head of TDS to help 
conduct the evaluation exercise. The team made a preliminary visit to the 
organization: This visit reconfirmed the apathy, demoralization, frustra- 
tion and lack of direction prevailing among the core team and animators. 
So, the process of evaluation was essentially to encourage the core team 
members to reflect on what they have failed to do and to visualize what 
relevant role they can play within the present situation. 


Three months after the preliminary visit, the evaluation process was 
carried out. The process continued in three phases, over a period of 12 
months. Both the facilitators were not involved during the whole period 
of the evaluation process. During each stage of the evaluation process, the 
facilitators raised the concerns and tried to put them in a perspective, and 
then left it to the organization to work it out in more detail so that they 
themselves could systematize their understanding. 


The prime objective of Phase I of the process was to identify the issues 
of evaluation. This exercise was carried out among the core group mem- 
bers only. The main contention was, who was to be involved in the 
process of evaluation? It was thought that as the primary concern of the 
evaluation was related to the organizational aspect i.e. lack of direction 
and sense of confusion about the nature and style of functioning, perhaps 
it was important to involve the core team at the beginning, and, as tne 
process moved on, the animators and selected people from the tribal 
communities could join. Perhaps at the beginning of the process the 
involvement of all the members could create and heighten tensions 
among the core members to reflect on their frustration and other concerns. 


The first phase of the process was also used to collectivize the concern 
among the core group members. It is important to note that all the core 
group members agreed on the concerns mentioned at the beginning. To 
recheck the degree of collective agreement on the issues, individual con- 
sultation and feedback was taken up by the facilitators. 
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After this exercise a weeklong exercise was undertaken with the core 
group members and animators, both collectively and individually to 
further reflect on these concerns. Certain subtle things like lack of norma- 
tive framework for work, accountability and delegation of responsibility, 
lack of clarity in the role of the governing body, lack of forum and system 
for decision making came up during individual consultations. 


The first two days were spent with the core group members. They were 
given enough support and strength to list out the achievements and 
failures. Once this exercise was over, the facilitators helped consolidate 
this information on the achievements and failures. During these two 
days, effort was also made to informally gét personal feedback on the 
tentative causes of failure and success and what each member felt about 
the future direction. This informal individual consultation helped the 
facilitators to situate TDS not only in terms of the present but also its 
future. This projection gave enough stimuli to the core staff to relate to the 
organization. 


After a gap of two days the core group and animators were advised to 
reflect on thé achievements and failures and to think about the future 
directions separately. The facilitators were constantly among the mem- 
bers to help the group to spell out the causes, to build up an atmosphere 
of sharing, to reduce personal conflicts, etc. 


The first phase of the exercise led to the identification of critical achieve- 
ments and major lacunae, limitations and shortcomings of TDS over the 
past years. The reflection process was separately carried out for the core 
team as there were some interpersonal conflicts and issues which needed 
to be addressed. This critical reflection process helped also in fhe team 
building for the core team. Separate sessions were held with animators to 
invite their inputs in the reflection process. The first phase of the critical 
reflection process led to the rearticulation of the directions and thereby 
clarification of the perspectives of TDS. Strengthening Tribal People’s 
Organizations (TPOs) became the major work for the future. | 


Soon after the process of reflection, the second phase of the evaluation 
started. The staff members of TDS carried the information systematized 
during the reflection process to the tribal villages. The purpose of this 
exercise was to analyse the emerging thoughts on the success, failure and 
future direction along with the tribal communities. During the next three 
months, the core staff and the animators decided to use village camps to 
evaluate their work with the tribals. These camps were organized in the 
form of village fairs, village meetings, group meetings, special camps for 
youth and women etc. Along with the feedback of the community, the 
core team was encouraged to develop various programmes and to think 


about the organizational mechanism of TDS that will help it accomplish 
the future direction. 


During the succeeding three months, the core staff also systematized 
the new information and insights they found during the village meetings. 


This process helped the core team to develop new programmes for the 
tribal areas and to strengthen TPOs. 
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The second phase of the evaluation process was culminated after six 
months of reflection, discussion and analysis by the core team and 
animators and their meetings with the tribals. A follow-up weeklong 
workshop was held with the core team and animators, separately and 
jointhy to look at the emerging programme proposals, The emerging prog- 
rammes of non-formal adult education, health care, biogas, land improve- 
ment, training etc. were critically examined in the light of overall perspec- 
tive of strengthening TPOs in the area. 


The core members have still not been able to locate their memberships 
in the organization as the roles and structure were not clear. Alongwith the 
development of programmes, organizational mechanism, role distribu- 
tion, work norms appropriate for effective functioning were also discus- 
sed. A more systematized planning and formal mechanism of functioning 
and procedure and norms necessary for its implementation were estab- 
lished during the meeting. 


During the next three months the core staff began to implement these 
decisions. At the end of it, a final session was held with the staff. This was 
to review the process of involvement of staff in the new direction and 
structure set by the team members themselves for the organization. 


During the whole process, the facilitators contributed essentially to the 
task of facilitating clarification of task, role and perspective by the core 
team of TDS. The final session not only revealed the revitalized strength 
of the members but also projected the future vision and the role of TDS. 


SOME FINDINGS 


The-central findings and main proposals of the organization are men- 
tioned here. 


A. ACHIEVEMENTS OF TDS 


Listed in order of priority and consolidated from different perspectives, 
following are the main achievements of TDS. 


1. Removal of moneylenders and liquor merchants 


This was seen as a pace-setter achievement which entailed removal of 
bondage and release of land, trees, utensils, etc., belonging to tribals. 


2. The emergence of TPOs 


The collective organization of tribals emerged as a vehicle for their own 
development. Tribal People’s Organizations (TPOs) were formed at vil- 
lage, regional and central levels and local tribal leadership emerged. 


3. Availability of institutional finance 


After the release of mortgaged land, the tribals made use of loans from the 
bank to acquire bullocks and other productive assets. More than 20 lakh 
rupees have been made available in the area and it helped build the 
credibility of TDS. 
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4. Savings schemes and consumption loans 
vidual and collective savings habit succeeded to 


The promotion of an indi , 
for consumption purposes became available to 


a large extent and loans 
avoid outside moneylenders in future. 


5. Enhancement in self-confidence of tribals 


The tribals have grown and have shed some fears. They have acquired a 
voice for themselves. Their actions and successes have led to an enhance- 
ment of their own confidence and self-image. They are now able to face 
outsiders squarely and discuss matters openly. 


6. Increase in the awareness among tribals 


The tribals have become more aware of their own situation, their needs, 
rights and responsibilities. There has been a significant reduction in de- 
pendence on black magic for health problems and a greater awareness of 
health rights. Similar increases in awareness of need for education and 
improved status of women have become visible. 


7. Physical achievements 


TDS has been able to provide assistance in physical areas like irrigation 
wells, drinking water wells, agricultural inputs, biogas, goatery, dairy, 
etc. There has been a reduction in the incidence of TB and malaria. 


8. The development of TDS 


From a small beginning, TDS has evolved into a viable, ongoing entity. It 
has developed external and local credibility, independence and self-confi- 
dence. It now stands on its own feet. It has been able to bring and hold 
together a committed group of young persons. 


There were several interesting variations in these achievements. Most 
animators saw mainly the tangible things as achievements. Even within 
the core group, some members emphasized tangible, physical accomplish- 
ments. Others primarily emphasized intangibles. The prioritization exer- 
cise done by animators, however, revealed striking similarities with those 
of the core group. The above listing is a coherent articulation of all these 
perspectives. 


B. FAILURES AND SHORTCOMINGS 


Though several items were detailed out as failures or shortcomings of 
TDS, the most important ones are listed below: 


1. Lack of an effective organizational system 


Problems like lack of working norms, procedures, salary structure, work 


follow-up, indiscipline, planning, accountability, etc., were mentioned in 
this regard. 


2. Lack of effective training 


Lack of staff development, reflection, study, identification and building of 
local talent, knowledge about how not to create dependence, develop- 
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ment of second line of leadership, etc., were identified as some of the 
issues here. 


3. Lack of future plans 


Overall strategy of TDS, its future directions and plans had become fuzzy 
lately. 


4. Communication and interpersonal difficulties in core group 


The question of centrality-marginality, sharing of information, lack of 
clarity about initiatives, mistrust, etc., were some of the issues raised. 


5. Plateau and slide-back 


Some areas of achievements of the past are being reopened as proble- 
matic. Less enthusiasm in TPO, more impact of outsiders and other nega- 
tive trends are visible. TPO has not been sufficiently involved in TDS. 


6. Lack of intensive work with youth and women 


Though sporadic efforts have been made to work with tribal youth and 
women, there has not been any pre-planned, intensive work with these 
two groups. This could have perhaps further strengthened TPO 


Many of these failures and shortcomings formed a focus of further 
analysis and discussions in the core group. Some were taken up as future 
directions for TDS. 


FUTURE DIRECTIONS 


The following future directions for TDS were evolved through consen- 


sus: — 


1. To strengthen TPO 

By strengthening village and regional committees, including TPO 
functionaries in the activities of TDS; encouraging TPO to take on more 
responsibility for the activities of TDS; TDS-initiated activities; training 
TPO leaders and functionaries. 


2. To build and consolidate organization of youth/women 

By organizing non-formal education, meetings, melas, training program- 
mes, camps, exposure to other areas and groups; inducting local youth to 
work with TPO as its animators; starting with some women health work- 
ers to develop nucleus of women’s organization. 


3. To enhance cultural exposure of tribals 

Culture as entertainment, source of solidarity, collective identity, way of 
binding themselves, means of expressing themselves, for raising aware- 
ness; as a mechanism for building new organizations and strengthening 
TPO; cinema slides, puppetry, folk dances, songs, etc., as possible ele- 
ments. 
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4. To organize primary and non-formal adult education 


Legal education as part of adult education, for mobilization and organiza- 
tion-building; formal primary education to be designed innovatively with 
the active involvement of TPO, as a long-term strategy for sfrengthening 


the tribal organizations. 


ISSUES 


Some interesting issues in this evaluation were: 


1. Building a climate of openness, trust, sharing and reflection in the 
early stages was a crucial intervention by the facilitators. The core team 
members were demoralised, suspicious, cynical and angry when the 
evaluation began. It was necessary to create a climate where members 
could distance themselves from the TDS, and look at it critically. Substan- 
tial interventions had to be made by the facilitators in building such an 
atmosphere in the core group. And sustaining such a climate demanded 
continuity of such interventions for quite some time. 


2. Another issue related to the selection of the two facilitators. The core 
team of TDS was in such a state then that it could not have accepted an 
outside facilitator easily. The head of TDS approached the two facilitators 
tentatively. The initial visit by the facilitators was largely seen as an 
occasion of testing of facilitators by the core team members. The actions of 
the facilitators had to be confidence-generating and trust-building in the 
early stages. Only when the core team members began to feel comfortable 
with, and trust, the facilitators, the actual reflection process could begin. 
As one of the core team members later commented, ‘We were looking for 
committed insider-outsider facilitators”’. 


WORKERS’ 
EDUCATION 
PROJECT 


A project for non-formal educational work with workers in the unor- 
ganized sector is being carried out in a big industrial town of Uttar Pradesh 
for the past five years. The project had its roots in the need for the new 
generation of workers, including women workers who were joining the 
workforce (largely coming from rural areas), finding their survival difficult 
and being unable to cope with the demands of their new situation. Infor- 
mal discussions with small groups of these workers highlighted the need 
to have a forum where workers could come and share their experiences, 
talk to each other and discover ways of enhancing their understanding of 
their new roles, their rights and duties and develop a perspective on the 
working class. An informal initiative had been taken five years ago, where 
in a small room near the workers’ colony young workers assembled in 
twos, threes, fives, tens, during evenings and holidays to talk with each 
other. 


This initiative was taken by a senior worker-educator who had been 
active with the organized sections of the working class earlier. Gradually 
some more workers began to volunteer their time in this centre which 
began to be known as the Workers’ Education Centre. After two years of 
ad hoc and informal functioning, a small project was prepared to provide 
opportunity for a more systematic training of workers about their rights, 
preparation of learning materials for them and creating other events 
which would help build solidarity and unity among these young workers. 
This project had an initial duration of three years and has shown certain 
concrete results. 


THE PROCESS 


Before this project was to come to its end, the donors requested for an 
evaluation of the experience, so that future programmes could be sys- 
tematically evolved. The intervening organization was approached by the 
project leader to assist in this exercise of critical reflection. It was evident, 
during the consultations with the project leader and the representative of 
the donor agency, that resources was not the issue in the evaluation 
process. The issue was clarifying the future directions and perspective of 
the Workers’ Education Project. 


It was proposed that the reflection process could be in two steps: the 
first step could be with the project team which by then comprised of three 
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full time staff and six regular volunteers. Subsequently, discussions with 
workers who use the centre and the project could also be undertaken. A 
five day critical reflection exercise was undertaken with the project team 
and an analysis of the situation of the workers was the starting point for 
this reflection. Based on that analysis, the need for an educational effort of 
the type that Workers’ Education Project represented was discussed. It 
was in this context that the concrete experiences of the past three years of 
this project were analysed and, strengths and weaknesses of the project 


elaborated. 


Some of these key strengths and weaknesses were then used as a 
starting point for discussion with five different small groups of workers 
who had been using the centre. This work was spread over a period of two 
weeks, since workers had different shifts, and arranging for them to meet 
together in small groups of eight to 12 was a major logistic problem. 


The critical reflection exercise with the workers was focussed upon 
their experiences within the project, their educational needs and the 
extent to which these needs were met and their visions about the future 
of the Workers’ Education Centre. 


With these elaborations, the core project team discussed together and 
developed certain directions for the future which then became the basis 
for evolving a three years project proposal. In the following pages, a 
summary of the reflection process with the project team and workers was 
presented and key issues highlighted. 


IMPLICATIONS FOR WORKERS’ EDUCATION 


(a) In this context, no educational effort can be neutral. It has to respond 
to the needs of the workers and has to be, therefore, clearly biased in 
their interests. 


(b 


~~ 


The learning needs as articulated by workers themselves are broad 
and vague, and many a time workers themselves do not see any need 
for their own learning and education. Hence, the content of their 
education has to be “‘directed”’ by the project team and other resource 
persons who should have a better appreciation of the educational 
needs of such workers. For example, most of the workers do not even 
have knowledge of their own factories and this could be promoted 
through an educational investigation exercise carried out by the work- 
ers themselves with the help of the staff. 


(c) However, the methodology of education has to be such that it pro- 
motes workers’ participation, provides support and listening, and 
creates warmth and companionship. It is in this context:that the 
informal and comforting atmosphere at the centre becomes crucial. 


(d) The resource centre for educational materials that is being evolved 
will be initially used by project staff and some workers during the 


course of their struggles. Ordinary workers will make use of the 
resource centre only much later. 


(e) Such an educational effort demands multiple roles of the educational 
staff. Many a time these multiple roles create tension. This may partly 
explain difficulties in attracting and retaining appropriate staff in the 
project as well as the need for continuous training, development and 
emotional support to the educational staff. 


(f) The class of young workers whose educational needs this project has 
attempted to meet cannot financially support the activities fully. In 
fact, it is unlikely that such educational efforts can ever be fully 
self-supporting, and not need any financial assistance from outside. 


In the light of the context mentioned earlier, several factors have 
affected the project strategy and, therefore, appear as the weaknesses 
of the project, as outlined in the section below. 


FACTORS AFFECTING PROJECT STRATEGY 


(a) Workers’ response 


The project has been continuously suffering from what is seen as a 
poor response from the workers to its various activities. There is a 
need to take a realistic expectation of workers’ response and the pre- 
sent time-frame appears to be too short to expect anything dramatic. 
Moreover, it is useful to distinguish between the interest, participation 
and responsibility taken by the workers. There is also a need to see the 
response of the activists differently from the ordinary workers. Work- 
ers’ response is also likely to be different in situations of action as 
opposed to reflection, as well as those involved in personal and collec- 
tive problem-solving. 


(b) Meaning of participation 


The project seems to indicate some confusion regarding the meaning 
of participation. This is reflected both in the management of the 
project as well as its activities. In the management of the project, 
participation may not imply equal.and identical activities and respon- 
sibilities by different members of the staff, volunteers, workers, activists, 
etc. The staff could have taken a more active role of leadership in 
determining programme content and frequency without necessarily 
reducing workers’ participation. 
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(c) Response to union 


P The project and its staff need to have different strategies to respond to 

; union vs. non-union situations. This is particularly important because 
many unions are just beginning to get formed and are likely to be 
sensitive to their territorial claims. 


(d) Linkage with struggle 


~~ 


Educational activities of workers cannot be seen in isolation of their 
struggles. This has been demonstrated time and again, through suc- 
cessful, educational efforts carried out during the various struggles 
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undertaken during the past three years. Perhaps the project has vacci- 
lated on this account of linkages with the ongoing struggles of the 


workers. 


(e) Project structure 


The present project structure has roots in its history. This has implica- 
tions on how the project is perceived by the workers. The behaviour 
of the full-time staff also affects workers’ perception of the project. 
The structure of an external steering committee and three full-time 
professional paid staff is likely to be a burden in the next phase where 
major responsibility and control of workers is envisaged. 


ISSUES 


The evaluation of Workers’ Education Project threw up some interesting 
issues. 


1. One of the major issues in the project was that of perspective. The 
centre has been initiated informally, and had not yet developed any clear 
position on its relationship. As the evaluation process moved forward, 
this issue of perspective became the main focus of deliberation. However, 
the initial focus of evaluation proposed by the project leader, as well as the 
donor representative, was to assess the programme and its impact. This 
evolution of the main concern of the project as the central focus of evalua- 
tion is an indication of the methodology itself. 


2. Another issue in the evaluation process was the mechanism to in- 
volve workers in the reflection process. Since most workers had shifts, 
worked in different places, and used the centre in different ways, it was 
quite a logistical and conceptual exercise to promote their involvement in 
the exercise. It was felt that workers could be involved through group 
discussions, planned at different times and days to suit their convenience. 
The reflection process of workers thus provided an important perspective 
to the evaluation of the project. 


ISSUES IN 
PARTICIPATORY 
EVALUATION 


Having looked at various examples in the preceding section, it is now 
possible for us to highlight some of the key issues that have emerged in 
the course of our practice and an elaboration of the methodology of 
Participatory Evaluation. These issues are by no means exhaustive but 
indicate the kinds of concerns that have regularly confronted us as well as 
Our partner organizations in the course of planning and implementing 
Participatory Evaluation efforts. Here we describe the issues, the different 
nuances and aspects, and attempt to state our own position on it as clearly 
as we understand it today. 


1. The first issue relates to the difference between methodology and 
method. 


Participatory Evaluation is a methodology of systematic and critical reflec- 
tion and analysis of our work. It is a methodology that has a certain 
world-view and philosophy, and the world-view of participatory evalua- 
tion is to enhance the capacity of ordinary people, projects and groups to 
carry out a systematic and critical reflection and evaluation process of 
their own activities, programmes, organizations and perspectives on a 
regular and ongoing basis. It is this demystification of evaluation and 
strengthening the capacity at the grass-roots to appropriate, control and 
utilize evaluation that reflects the basic premises of the philosophy of 
Participatory Evaluation methodology. It is within this overall methodol- 
ogy that the choice of tools, technique and methods can be understood. 
Should a particular evaluation exercise require, these methods can be for 
data collection, for facilitating reflection, and for improving analyses. That 
is to say a whole range of tools, techniques and methods can be utilized in 
a specific situation in order to facilitate a concrete reflection and evaluation 
exercise. 


2. an thought you were qualitative” 


A very common comment that we hear is that Participatory Evaluation 
implies qualitative information and its analysis. Wherever we come up 
with the need to collect or analyse quantitative information in the course 
of a given Participatory Evaluation exercise, it is resisted, as if this isa 
violation of the norms of Participatory Evaluation. 


The basic confusion mentioned in the previous issue is the source of 
such. misunderstandings. The type of information that is necessary to 
carry out a systematic process of reflection and critical analysis depends 
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on a given Participatory Evaluation exercise and its specific objectives. 
That information can be statistical, cognitive, perceptual, qualitative, etc. 
Depending on the type of information we need, the type of data collection 
methods will have to be created. It is conceivable to collect (and we have 
used it as you can see from the case studies) qualitative as well as statistical 
and quantitative information in the course of an evaluation exercise. 
When we need such information, we have to create instruments and 
questionnaires that are able to elicit such information. We also need to 
look at records and documents which contain such information, both 
inside and outside the organization. Qualitative information is more easily 
obtained through open interviews, group discussions, field observations, 
etc. 


Thus the choice of a particular set of methods for data collection, be they 
interviews or questionnaires or group discussions or instruments of ob- 
servation, depends largely on the specific objectives of a given Participat- 
ory Evaluation exercise. Those objectives decide what kind of information 
we need to collect; and the type of information we need to collect deter- 
mines the kinds of information collection methods and tools we are going 
to use. 


As the case studies seem to indicate, one can make some tentative 
generalizations about the patterns viz.-a-viz. the method of data collection 
for different types of Participatory Evaluation exercises. Clearly where the 
focus of the evaluation is on the programmes and their impact on the 
ground, a more detailed information about impact can be collected only 
through structured instruments and questionnaires, and collection of 
statistical information from records and documents. Thus, Participatory 
Evaluation exercises initially and largely focussing on programmes tend 
to be utilizing methods of information collection that are highly quantita- 
tive and structured. 


On the other hand, those Participatory Evaluation exercises where the 
focus primarily is on clarifying, sharpening or modifying perspectives 
and directions of an organization, its teams, people’s movement, etc.,. 
much more interactive and dialogue methods of information collection 
and analysis have to be utilized. Shifts in perceptions and understandings 
are necessary in order to bring about sharpened and modified perspec- 
tives within an organization and indepth interviews, informal, small 
group discussions, dialogues, and other interactive and open-ended 
methods of information collection and analysis become important in such 
an exercise. Obviously information about programmes and their impact 
helps in clarifying the perspective as well in certain situations, but de- 


pending on the specific situation, the methods of data collection utilized 
can vary considerably. | 


When it comes to the focus on the organization of a grass-roots project 
or an NGO ora people’s movement as the primary focus of a Participatory 
Evaluation exercise, information of various types becomes necessary. It 
becomes necessary to understand the manner in which organizations 
function and therefore perceptual, attitudinal information may be as im- 
portant as information available from documents and records and files. 
So, a combination of types of information may demand a combination of 


types of data collection methods to be utilized in the course of a Participat- 
ory Evaluation exercise. 


There is one other aspect in the issue of methods of data collection and 
that has to do with the potential of the method to catalyse reflective 
processes among the members of the organization itself. A set of methods 
which are traditionally not considered methods of information collection 
and analysis like role plays, case studies, drawings, audio-visuals, etc., 
may be necessary in order to catalyse reflection among the local people 
and field workers since those are the methods that can be easily utilized 
by them. For a largely illiterate or semi-literate population, use of paper 
and pencil instruments may be not so effective. So the choice of data 
collection methods, wherever flexibility permits, can be such that they 
facilitate greater involvement and participation, in the reflection and 
analysis process, of the members of the organizations and the group 
whose involvement is critical in the Participatory Evaluation exercise. 
Obviously this choice has to be made within the context of a specific 
Participatory Evaluation exercise based on the specific objectives of evalu- 
ation. No general prescriptions can be made here as well. 


3. Who is participating in Participatory Evaluation? 


Like in other areas, the word participation in the context of Participatory 
Evaluation is also a much confused, abused and misused word. The word 
“participation’’ sometimes provokes visions of mass involvement, border- 
ing on anarchy. In our definition of Participatory Evaluation as stated 
previously, who participates depends on a specific situation and on the 
concrete objectives of the Participatory Evaluation exercise. If we take a 
grass-roots non-governmental organization’s field programme and if we 
look at ‘the focus of evaluation as programmatic, then clearly most crucial 
actors in the evaluation exercise would be field workers of the NGOs and 
the local people and beneficiaries. They would provide the initial analysis 
and information which can then become the basis for reflection by other 
members of the organization. 


If, however, the focus of evaluation is clarifying the perspective of the 
NGO then the involvement of members of staff of the NGO is more 
important than involvement of local population. Changes in perspectives 
as well as in organizational form, structure and practices do require the 
involvement, commitment and support of key members of any organiza- 
tion and, therefore, an identification of such key members is necessary if 
this process of reflection, analysis and change has to be facilitated. Thus 
the participants in the examples of Participatory Evaluation given here 
vary considerably in their mixture, character, groupings, etc. Within a 
given Participatory Evaluation exercise, different parties can be involved 
at different stages as well as in different ways, and it is not necessary that 
every party who is a stake-holder in the Participatory Evaluation exercise 
gets involved in the same way. 


The concept of stake-holders may be useful in understanding who gets 
involved in a given Participatory Evaluation exercise. The party whose 
interests are directly and primarily influenced by the focal objectives of a 
given Participatory Evaluation exercise must be involved. But it does not 
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mean that all the parties must be involved throughout in an identical 
fashion. This distinction needs to be understood and-clarified. 


The issue about the nature of stake-holders and who participates also 
gets influenced by the extent and depth of a given Participatory Evalua- 
tion exercise. It is possible that a particular localized programme of a large, 
well-established NGO with clear perspective and effective organization 1s 
the focus. In which case, field workers and staff involved in that prog- 
ramme and local population and groups benefiting from the same are the 
ones who require to play key roles in such an evaluation exercise. It is 
possible in such a case that other segments of the organization may not 
become actively involved. It is also conceivable that in a large non-gov- 
ernmental organization where the primary focus of evaluation is the 
organizational mechanism and organizational issues, the,starting point 
could be the governing board, the executive head and his/her key team at 
the very top of the organization, and for quite sometime they may remain 
the key participants in the process, while other segments of the organiza- 
tion, field level workers and local populations may not be involved in the 
evaluation exercise directly. 


The key question is how do we identify actors, parties, individuals, 
segments inside and outside the organization which have a varying con- 
tribution to the critical reflection and analysis process? It is this kind of 
focussed identification that should help us determine participants in a 
Participatory Evaluation exercise. Thus an omnibus, open-ended state- 
ment that everybody must participate since it is Participatory Evaluation 
makes no sense. Once again, a definition of Participatory Evaluation is not 
based on participation per se, but on its capacity to initiate, build and 
sustain a critical reflection process in a given organization, programme or 
movement. 


4. But what about the donors? 


A common question in our experience has been the role of donors in a 
Participatory Evaluation exercise. As we have mentioned in the case 
studies many a times, a particular donor or a team of donors took the 
initiative in convening a process of Participatory Evaluation; sometimes 
this initiative comes jointly through the programme or NGO funded by 
them. Sometimes, it comes directly to a facilitating agency. The needs of 
resource providers in terms of evaluation have to be recognized as legiti- 
mate. This is true in all situations. The question is what kind of informa- 
tion do a set of resource providers need from a given context in order to 
help them make their own decisions. Thus the concerns and needs of 
donors must be explicitly taken into consideration and the place for this 
consideration to be articulated is in the initial stage of setting the objectives 
of evaluation. Inputs on resource providers and donors can be useful in 
fixing the objective of a given Participatory Evaluation exercise. 


The other place where donors and resource providers can contribute is 
at a stage where the critical analysis and reflection process has progressed 
somewhat and broad findings are being articulated and future directions 
are being evolved. Because of a certain type of knowledge and under- 
standing that resource providers have of a given project or a programme 


or an organization, they can also help in this process of reflection as well 
as setting of future directions and programmes. 


_ But there are several difficulties that arise in practice in the donor’s role 
in a given Participatory Evaluation exercise. First of all, we have tried to 
de-link funding from the evaluation. We have tried to ask the donor and 
the project to discuss their funding needs separately. In many cases, the 
donors have assured continuity of funding ang the evaluation exercise 
could help in determining what type and what form and for what purpose 
the funding may take place. 


De-linking evaluation from funding ensures that the process will be 
genuine, authentic and critical. If a grass-roots organization or people’s 
movement is worried about continuity of its funding as a consequence of 
its evaluation exercise, it will try to paint a rosy picture, will try to avoid 
critical reflections and ‘may end up covering up key issues rather than 
confronting them. 


In the case studies mentioned previously, there are examples where an 
organization on its own decided to initiate an evaluation process without 
any direct or indirect request from the donors. This of course is one of 
most healthy possibilities because this then keeps the issue of funding 
outside the purview of the evaluation process, and creates the possibility 
of the process being authentic, open and genuine. As a facilitating agency, 
it has been our endeavour that we do not become agents or hatchet men 
for the donors; we have tried to distinguish ourselves from the donors; we 
have encouraged the project or the organization to discuss openly the 
need for an evaluation at that point in time; and, there have been examples 
where this discussion has led to a decision by the project holders or the 
NGO not to continue with the evaluation exercise at.that point in time. 
Thus even when a request to us comes from the donor, we try to establish 
links with the project/NGO, encourage them to assert their interests and 
to play an active and controlling role in the evaluation process. We believe 
that we should behave as facilitators of the reflection process of the 
project or the NGO, and not managers of a process on behalf of th 


donors. : 


5. What is the role of outsiders? 


In our context, the debate on outsiders is a perennial one. First of all, it is 
hard to figure out who is an outsider. The issue of the outsider in a 
Participatory Evaluation exercise gets posed in the context-of subjectivity 
and objectivity. Many people believe that a project, an organization or 
movement on its own cannot be critical enough of its own experiences, 
practices and programmes, and thatiits self evaluation may be biased,.and 
that it will hide weaknesses and project strengths. In the very definition 
of Participatory Evaluation, this is.a false notion. 


If a project, a grass-roots organization, a people’s movement is itself 
interested in critically reflecting on its own experiences, practices and 
organization, then it is in its own interest to make the process critical, 
reflective and open, and not to hide or gloss over shortcomings or weak- 
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ubjective and biased, be- 


sses. So our concern is not whether this is $ ed, 
nee ay or the other, subjective 


cause in our view, all such efforts are, in some W 
depending on who the subject is. 


The issue is, can, without any external assistance, a project, a group, a 
movement or an organization facilitate the entire process of critical reflec- 
tion on its own? Will it have the capacity, the competence and the re- 
sources to ensure that appropriate and relevant parties and individuals 
are involved in setting of the objectives, collection of information, in 
analysis of their information and in acting on the basis of that analysis? It 
is our experience, as well as our contention, that it is possible in certain 
circumstances. We have seen that many well organized groups, institu- 
tions, movements have this capacity and competence within themselves 


to carry out, on an ongoing basis, the process, of critical reflection. 


It is also possible that some groups, projects or organizations may not 
have this capacity and that is where an outside agency or individual could 
facilitate this process. The role of the outsider in such a situation could be 
to provide a distance, to help raise issues and questions which may 
otherwise not get raised or may be difficult to raise, to bring into the open, 
information and concerns which are generally not so clearly stated, to 
help in-articulating the objectives of an evaluation, to help create methods 
of information collection and analysis and to help the project or the 
organization take charge of its own evaluation and use it in its own 
interest. 


There are occasions where we have also played the role of provider of 
inputs on a particular issue, where we have brought in information or a 
resource person from outside to share what is happening elsewhere as a 
contribution to the process of reflection. But in all its diverse roles and 
activities, the outside agency must continue to ensure that the project or 
the organization “owns” the evaluation process, feels it is its own and for 
itself, and gets involved in taking responsibility for it. This does not rule 
out the possibility of managing, partially, the process of evaluation, par- 
ticularly the process of data collection. | 


We have found that in an ongoing organization, members of the group 
are already over-committed; busy and want us to take the responsibility 
to ensure that the decisions jointly made with them get implemented 
(viz.-a-viz. collection of information and.its analysis). There are situations 
where an organization deputes a team of its own to manage the process 
and we essentially play the facilitator’s role, 


Thus this specific role of the outside agency orindividual varies consid- 
erably, depending on the given situation and the specific objectives for 
the evaluation exercise. Any simplistic and universal prescriptions about 
the outsiders’ roles are not warranted in such circumstances. 


A related role that we have played on most occasions is that of preparing 
the report. Invariably, grass-roots organizations have felt that they need a 
document for future reference and for use inside and outside the organiza- 
tion, which reflects the key findings and the major decisions for follow-up 
made in an evaluation exercise. It must be stated here that a written 


document or an evaluation report is not a necessary ingredient of any 
Participatory Evaluation exercise. It is essentially a record of the process 
and the outcome of the exercise, for use by the organization primarily, and 
by others if it can be a learning experience. 


We have found that preparing a draft report on the basis of initial 
analysis which is jointly shared, helps us to also put our ideas and analysis 
in place. Invariably, this draft report is then sent back to the organization 
which: discusses it among themselves and then is discussed jointly with 
the facilitating agency. These discussions on the draft report provide 
further insights which then become the basis for planning future direc- 
tions, both of which are incorporated in the final report. We believe that 
this report is for the organization, and that then it is the'responsibility of 
the organization to decide whether or not to share it with others. 


There are also examples of keeping the record of an evaluation experi- 
ence in audio-visual form. This needs to be further developed and im- 
proved. 


6. The continuity of evaluation 


In our definition, a Participatory Evaluation exercise should be seen as an 
ongoing process of critical reflection within an organization, programme 
or people’s movement. Therefore, it is continuous as well as periodic. 
After a few years of experience in one area or in one programme or with a 
certain methodology, it is possible to take time off to reflect on it critically. 
And this may become an issue-based or event-based evaluation exercise. 


Our experience has suggested that the follow-up of the Participatory 
Evaluation exercise begins to take place during the exercise itself. In 
almost all the case studies mentioned here, changes in programmes, 
perspectives and organizations began to be decided upon and im- 
plemented during the course of the evaluation process itself. And some- 
times the report of the evaluation exercise is essentially a summation of all 
the decisions made and implemented in the process itself. 


Besides, it is also desirable to concretely plan steps for follow-up from 
the evaluation exercise. In mary cases, broad directions are agreed upon 
and it is here that future steps in planning and implementing those 
directions need to be set up at the end. of ‘a Participatory Evaluation 
exercise. The role of the outside facilitator or facilitating agency can cton- 
tinue in the follow-up period, but perhaps needs to be re-negotiated. It is 
our experience that facilitating a critical reflection exercise demands one 
type of role; and providing input to implement a new programme or 
create new organizational mechanisms or systems demands quite another 
role. Thus we see our role as facilitators of an evaluation process terminat- 
ing with the making of the follow-up plans and we may continue to be 
involved, but in a different capacity and different role, in implementing 
those follow-up plans. This possibility of renegotiation also opens up the 
choice for the organization to involve other resource persons, resource 
organizations and expertise from outside, should it so desire. 
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On the whole, the entire set of ideas and practices related to Participat- 
ory Evaluation are fairly exciting and creative. We find ourselves continu- 
ously challenged and energized in each and every evaluation exercise. 
Ultimately, the criteria for judging its effectiveness have to be the experi- 
ences of those who have initiated and gone through this process. If they 
have benefited through this process of reflection in some way Or the other, 
then it has contributed to enhancing their capacity for continued work. If 
they have not, then it has failed in its objectives. As an emerging practice 
and evolving methodology, much more needs to be shared and analysed 
about its experiences in India and elsewhere. We hope that this would 


encourage others to do the same. 
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The Society for Participatory Research in Asia (PRIA), New 


» Delhi is a non-profit voluntary organization registered 
under the Indian Society’s Act. 


Participatory Research is a methodology based on the 
belief that knowledge is power and therefore contributes 
towards the empowerment of the oppressed and the poor. 

It promotes the involvement of the poor and theirorganiza- 
tions and representatives in the creation and utilization of 
knowledge in their own collective interests. Thus, it at- 
tempts to challenge the monopoly over knowledge and its 

tools in the hands of the few. | 


PRIA works with local groups and activists involved in 
the education and organization-building efforts and strug- 
gles of the marginalized and underprivileged sections of — 
society. ae 


We provide support through research, training, evalua- 
tion, networking with groups on common Issues and pre- 
paration and dissemination of learning materials. 


We are a team of 15 people working with several partner < 
_ groups all over India. | : 


In the last six years of its existence, PRIA has specially 

focussed on primary health care and adult non-formal edu- 

» cation, problems of deforestation, land alienation and large 

dams, women and work, women and sanitation, occupa- 
tional health hazards and management issues of NGOs. 


PRIA is constantly involved in efforts to promote the 
understanding and practice of PT philosophy and 
methodology by holding workshops and documenting sev- 
eral efforts in the country. oe 
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